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Glossary of Terms 

Merck Institute for Science Education Partnership — Created in 1993 by Merck & Co., 
Inc., MISE began a 10-year commitment to the goal of raising student interest, participa- 
tion, and performance in science. MISE formed partnerships with school districts in 
Linden, Rahway, and Readington Township in New Jersey, and North Penn in Pennsyl- 
vania. 

Leader Teacher Institute (LTD — Launched in 1995 to provide intensive professional 
development to a select group of teachers from each partner school over a three-year 
period. These teachers would then become the Leader Teachers within their schools. 

Leader Teacher — Selected teachers who attended LTIs and worked with new teachers 
by orienting them to the new module-based science curriculum and provided instruc- 
tional guidance and support. 

Peer Teacher Workshops (PTWs) — Launched by MISE in 1996, PTWs provided profes- 
sional development opportunities open to all K-8 teachers in an effort to engage more 
teachers in science reform. PTWs were open for voluntary enrollment and each was led 
by a team consisting of a combination of Leader Teachers, content specialists, instruc- 
tional specialists, and classroom teachers. 

Principals' Institutes — MISE offers Principals' Institutes to make sure that principals 
are remaining informed about, and support, inquiry-based instruction and other aspects 
of the reform process. 
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Introduction 

With an initial 10-year commitment 
from Merck & Co., Inc. the Merck Insti- 
tute for Science Education (MISE) was 
founded in 1993 to demonstrate that 
virtually all students could reach high 
levels of scientific literacy. Shortly there- 
after, MISE formed partnerships with 
four public school districts — Linden, 
Rahway, and Readington Township in 
New Jersey, and North Penn in Pennsyl- 
vania — where Merck has major facili- 
ties and a history of providing employee 
volunteers and supporting local science 
education initiatives. These district 
partnerships quickly merged into one 
multi-district Partnership. 

MISE's approach to improving 
science teaching has been systemic, 
addressing both policy and practice in 
the partner districts. MISE has helped its 
partners plan strategically, select high- 
quality instructional resources, support 
teacher learning, and carry out instruc- 
tional and curricular reforms. Working 
together, MISE and its partner districts 
have developed and implemented a 
shared vision of good science instruction 
based on national and state standards. 
This vision has been the basis for the 
design and delivery of professional 
development for teachers and adminis- 
trators. Sustained by the Partnership 
since 1994, this professional develop- 
ment program has helped the partner 
districts make significant reforms in the 
teaching of science and mathematics. In 
1993, MISE contracted with the Consor- 
tium for Policy Research in Education 
(CPRE) at the University of Pennsylva- 
nia to evaluate the effectiveness of its 
work with the four districts. CPRE has 
documented the activity and progress of 
the Partnership for a decade and has 
issued a series of reports on its impact.^ 



^ See Appendix A and B for a complete listing 
of reports. 



In this report, we assess the 
Partnership's approach to professional 
development. Specifically, we address 
the following questions: 

1 . How has the Partnership's profes- 
sional development measured up 
against the emerging standards for 
professional development? 

2. Has participation in Partnership 
professional development resulted in 
increased teacher content knowl- 
edge? 

3. Has participation in Partnership 
professional development led to 
changes in instructional practice? 

4. Has participation in Partnership 
professional development resulted in 
improved student achievement? 

5. Has MISE's strategy strengthened 
district capacity to support the 
improvement of teaching? 

6 . What lessons can be learned from 
the experience of MISE and the 
Partnership with professional devel- 
opment? 

The report is organized into eight 
sections. The introduction describes the 
research questions and data collection 
methods used by CPRE. In the second 
section, we summarize findings from 
recent studies of professional develop- 
ment, and describe the consensus stan- 
dards of quality that have emerged in 
the past decade. MISE's theory of action 
and the professional development pro- 
vided by the Partnership are described in 
section three. In section four, the latter is 
assessed against the consensus standards 
of quality. Section five presents an ex- 
amination of the impact of the Partner- 
ship professional development strategy 
on the capacity of the partner districts 
and schools to support reforms in sci- 
ence education, and in the next two 
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Table 1. 


Enrollments and Grade Spans of the Partner Districts, 
1993-1994 to 2000-2001 


District 


K-8 Enrollment 
1993-1994 


K-8 Enrollment 
2000-2001 


Grade Span 


# of Schools, 
Grades K-8 
2000-2001 


Linden 


3,405 


4,204 


K-12 


10 


North Penn 


7,759* 


10,534 


K-12 


14 


Rahway 


2,229 


2,747 


K-12 


5 


Readington 


1,473 


2,113 


K-8 


4 



* This figure represents grades K-7 as eighth graders attended the high schooi. 



sections, we look at the Partnership's 
impact on teaching and learning. We 
conclude by examining some lessons to 
be learned from the experience of MISE 
and the Partnership, and consider the 
issues that it raises. 

Methodology and Data 
Sources 

The data for this report come from a 
longitudinal evaluation of a business-led 
partnership seeking to improve the 
quality of science instruction in grades 
K-8 in four school districts. Two of the 
districts served small urban communi- 
ties; one was suburban, and one was a 
rapidly developing rural area. In Table 1, 
enrollment data are presented for the 
1993-1994 and 2000-2001 school years, 
representing the beginning and near the 
end of the study period. 

Table 2 displays data on minority 
enrollment and the percentage of stu- 
dents eligible for free and reduced-price 
lunch. There is considerable variation 
among the partner districts, which range 
from blue-collar towns in New Jersey to 
rural and suburban enclaves. The data 
show increases in minority enrollments 
and the numbers of students living in 
poverty in both Linden and Rahway. 

Thirty-four schools served grades K-8 
in the four districts. The average school 
population of the schools was 551 



students. The largest school had 1,380 
students and the smallest had 284 
students. Eighty percent were elemen- 
tary schools and 20% were middle 
schools. 

Eor a decade, CPRE has documented 
the work of MISE and the collaborative 
it formed with its four partner districts, 
known as the Partnership. The CPRE 
evaluation team has examined the 
evolution and efficacy of the theory of 
action guiding this work, focusing on the 
quality and impact of the professional 
development and technical assistance 
provided by MISE and its partners. We 
have looked for evidence of impact on 
district and school policy, school culture 
and organization, curriculum and 
classroom practice, and student out- 
comes. Between 1996 and 2002, the 
Partnership was funded in part by a 
Local Systemic Reform (ESC) grant from 
the National Science Eoundation (NSE), 
and CPRE cooperated with Horizon 
Research, Inc. (HRI), the national evalu- 
ator of the ESC program. HRI provided 
some of the interview protocols, an 
observation protocol, and a teacher 
survey. HRI also drew annual samples of 
teachers to be observed. However, with 
support from MISE, CPRE broadened 
the HRI data collection activities to 
survey all K-8 teachers and administra- 
tors in the four districts and to conduct 
additional observations and interviews. 
As a consequence, each year CPRE staff 
conducted interviews with district and 
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Table 2. Minority Enrollments and Poverty in the Partner Districts, 

1993-1994 to 2000-2001 


District 


% Minority 
1993-1994 


% Minority 
2000-2001 


% Eligible for Free 
and Reduced- 
price Lunch 
1993-1994 


% Eiigibie for Free 
and Reduced- 
price Lunch 
2000-2001 


Linden 


46.6 


61.0 


30.0 


43.5 


North Penn 


12.7 


18.3 


NA 


10.5 


Rahway 


47.4 


65.0 


22.0 


30.0 


Readington 


3.5 


5.2 


NA 


1.2 



school staff, professional development 
leaders, and MISE staff, observed class- 
room teaching and professional develop- 
ment activities, and administered the 
HRI surveys. On average, 600 to 700 
teachers completed the annual HRI 
surveys and response rates ranged from 
70% to 85%. Between 20 to 100 class- 
room observations and 50 to 100 inter- 
views were conducted each year. A set of 
case study schools was visited every two 
years. 

CPRE has produced annual reports 
on the work of MISE and the Partner- 
ship.^ Also, CPRE issued a series of 
reports on the long-term impact of MISE 
and the Partnership including this report 
on professional development.^ 

Research on 
Professional 
Development 

Policymakers and school reformers 
are calling for improvements in the 
quality of teaching, and there is broad 
agreement that this requires more and 
better professional development for 
teachers. Linda Darling-Hammond 
(1997) of the National Commission on 
Teaching and America's Euture has 



^The eight annual reports are listed in 
Appendix A. 

® These longitudinal reports are listed in 
Appendix B. 



recommended improving both the 
quality and quantity of professional 
development in order to promote the 
continuous improvement of teaching. 
Elmore (2002) argues that both the 
pedagogical skill and content knowledge 
of professional teachers must be en- 
hanced to produce the sustained gains in 
student learning envisioned by the 
advocates of standards. Yet, in spite of 
these and similar recommendations from 
many other professional organizations 
and reform leaders, as well as the pres- 
sures emanating from higher-stakes 
accountability systems, at the beginning 
of the 21®* century, most teachers in the 
United States experience limited oppor- 
tunities to improve their knowledge and 
skill (Desimone, Porter, Caret, Suk Yoon, 
& Birman, 2002). And these professional 
development opportunities usually are 
fragmented, poorly aligned with the 
curricula teachers teach, and inadequate 
to meet their needs for deeper knowl- 
edge of subject matter and understand- 
ing of pedagogy (Cohen & Hill, 2001; 
Corcoran & McDiarmid, 2000; 

Desimone et al., 2002; Elmore, 2002; 
Loucks-Horsley, Love, Stiles, Mundry & 
Hewson, 1998; Supovitz, 2003). Eor too 
many teachers, professional develop- 
ment opportunities consist of occasional 
"in-service programs" of limited dura- 
tion — typically half-day or full-day 
workshops in which they are exposed to 
faddish teaching strategies or new 
theories about teaching and learning. 
Often, the content of these programs is 
not connected to their daily work, and 
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there is little or no follow-up support to 
help them use the knowledge or skills 
they have acquired. 

In recent years, researchers have 
begun to examine what works in profes- 
sional development. Supovitz, Mayer, 
and Kahle (2000) studied the effects of 
intensive, standards-based professional 
development on teachers of science in 
Ohio and found that teachers became 
more positive about instructional re- 
forms and more likely to use inquiry- 
centered pedagogy. Cohen and Hill 
(2001) studied mathematics teachers 
who participated in the intensive cur- 
riculum-based professional development 
offered by the California Subject-matter 
Network in the 1990s, comparing the 
changes made in their practice with 
those made by teachers who received 
more conventional professional develop- 
ment in mathematics. They found that 
teachers who participated in the former 
were much more likely to make changes 
in their practice, and that these changes 
were associated with gains in student 
performance. They concluded that 
providing teachers with extended con- 
tent-specific opportunities combined 
with follow-up support produced re- 
sults. 

These findings are consistent with 
those from other studies of the California 
Subject-matter Network (Pennell & 
Firestone, 1996; Wilson, 2003), research 
on the reforms in literacy and math- 
ematics teaching carried out in District 
#2 (Elmore & Burney, 1997), evaluations 
of the National Writing Project (St. John 
& Stokes, 2003), and secondary analysis 
of data from NSF's ESC projects 
(Supovitz & Turner, 2000). Moreover, 
studies of comprehensive school reform 
programs like Success for All and 
America's Choice (Haslam & Seremet, 
2001; Supovitz & Taylor, 2003) and the 
national Eisenhower program 
(Desimone et. al., 2002; Birman, 
Desimone, Caret, & Porter, 2000) have 
yielded similar results. The general 



conclusion is that extended opportuni- 
ties to engage in professional develop- 
ment that is aligned with the curriculum 
to be taught, and accompanied with on- 
site follow-up support, can produce 
significant changes in classroom practice 
and benefits for students. 

These findings are consistent with 
the experiences of those who design and 
deliver professional development, and, 
as a consequence, a consensus has 
emerged about what constitutes effective 
professional development. Researchers 
and designers of professional develop- 
ment have reached broad agreement on 
a set of principles or standards that they 
believe characterize effective profes- 
sional development programs; that is, 
programs leading to changes in teaching 
practices. 

According to this "consensus" view, 
high-quality professional development 
programs: 

• Are grounded in research and clini- 
cal knowledge of teaching and 
learning; 

• Are aligned with the curriculum and 
assessments in use in the setting; 

• Are focused on student learning in a 
particular context; 

• Model good practice throughout the 
program, modeling methods of 
constructivist teaching or inquiry- 
based methods; 

• Are active learning opportunities for 
teachers including practice, feed- 
back, and reflection; 

• Are transparent about the limitations 
of the evidence supporting the 
desired practice; 
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• Are intensive and include follow-up 
and on-site support for teachers; 

• Are sustained over time, allowing 
teachers to integrate new knowledge 
and strategies into their practice and 
to reflect on the experience; 

• Focus on building teachers' peda- 
gogical skills and content knowledge 
in their subject areas; 

• Utilize the expertise of teachers in 
schools and districts, cultivate lead- 
ers, and involve teachers in plan- 
ning; 

• Provide structures that facilitate 
collaboration among teachers both 
within and across schools; and 

• Respect teachers as professional, 
adult lifelong learners. 

(Cohen & Hill, 2001; Corcoran, 1995; 
Desimone et al., 2002; Elmore, 2002; 
Lieberman & Wood, 2002; Loucks- 
Horsley et al., 2003; National Research 
Council, 1996; National Staff Develop- 
ment Council, 2001; Supovitz, 2003) 

These 12 principles are consistent 
with the standards for professional 
development adopted by the National 
Staff Development Council (2001; 

Killion, Munger, Roy, & McMullen, 

2003), the American Federation of 
Teachers (1999), and the National Insti- 
tute for Science Education (Loucks- 
Horsley, Stiles, & Hewson, 1996). They 
reflect the findings from well-designed 
research studies as well as the clinical 
expertise of those who have designed 
and delivered professional development. 
While designing professional develop- 
ment according to these principles will 
not guarantee success, it can increase the 
likelihood that participants will use 
what they are learning to change their 
practice. We will use these principles as 



a framework for assessing the quality of 
the professional development provided 
by MISE. 

The MISE Theory of 
Action 

Guided by a board of advisors that 
included representatives of the National 
Science Resources Center, the National 
Science Teachers Association, the Na- 
tional Academy of Sciences, and leading 
scientists and science educators, and led 
by a respected science educator, MISE 
developed partnerships with educators 
in four districts in New Jersey and 
Pennsylvania aimed at reforming science 
teaching in the elementary and middle 
grades.'^ MISE sought a dramatic trans- 
formation from textbook-based, memori- 
zation-oriented instruction to guided 
inquiry in which students actively 
engaged in science investigations based 
on structured curriculum units such as 
those developed by FOSS (Full Option 
Science System) and STC (Science and 
Technology for Children). While some 
reformers subscribe to a more radical 
view of inquiry in which students deter- 
mine the topics and questions to be 
explored and conduct their own inquir- 
ies, MISE advocated a more practical 
vision suited to K-8 teachers. While 
supporting student inquiry, the MISE 
strategy used tested standards-based 
curriculum units as the backbone of the 
science curriculum. These units framed 
classroom inquiry and ensured that all 
students learned key concepts. MISE 
refers to this strategy as inquiry-centered 
science teaching. 

MISE sought to make inquiry an 
integral and regular part of the experi- 
ence of all students at all grade levels. In 



'‘A complete account of the story of MISE's 
partnership with the four districts is found in 
Corcoran (2003). 
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inquiry-centered classrooms, students 
were expected to conduct scientific 
investigations under the guidance of 
their teachers; learn to collect, analyze, 
and interpret data; and explain their 
observations and results. This vision of 
high-quality science education required 
moving elementary science teachers 
away from their dependence on text- 
books and this posed significant chal- 
lenges for many teachers. As in most 
school districts across the United States, 
elementary science in the four partner 
districts was taught by generalist teach- 
ers whose science backgrounds were 
limited — typically consisting of only 
one or two basic college courses. In fact, 
making sure science was taught at all in 
the elementary grades was a challenge 
when MISE began its work. There were 
no state assessments in science in 1993, 
and the reputations of elementary 
schools and their relationships with the 
state departments in New Jersey and 
Pennsylvania depended largely on the 
performance of students in reading, 
writing, and mathematics, not science. 
Thus, in many elementary classrooms, 
science, when it was taught, was often 
merely an extension of the reading 
program. 

To put their vision of science teach- 
ing into practice, MISE staff helped the 
four districts review and select commer- 
cially available science modules that 
could support student inquiry and cover 
the key concepts identified in state and 
national standards. To implement this 
new curriculum and engage in inquiry- 
centered teaching, teachers needed 
professional development, but other 
changes were needed as well. As a 
consequence, MISE took a systemic 
approach in its work with the partner 
districts, and its theory of action in- 
cluded the following 10 components: 

• Persuading districts to make the 
improvement of science teaching a 
priority, and to engage in serious 
planning to address it; 



• Developing a leadership team in the 
district that shared a common vision 
of science teaching grounded in 
inquiry and consistent with state and 
national standards; 

• Helping districts develop new cur- 
riculum frameworks for science, 
select appropriate instructional 
materials, and develop systems for 
the management of the materials to 
ensure they could be used effectively; 

• Providing access to content-based 
professional development for teach- 
ers over an extended period of time 
and on-site follow-up support to 
enable teachers to use the curriculum 
materials effectively; 

• Building district capacity to plan and 
deliver this professional develop- 
ment; 

• Supporting the use of assessments 
that were consistent with the vision 
of good teaching, including forma- 
tive assessments, end-of-unit assess- 
ments, and district-wide perfor- 
mance assessments and examina- 
tions; 

• Developing leadership cadres who 
could carry out this work; 

• Developing professional cultures for 
the districts and schools that would 
promote and support continuous 
improvement of science teaching and 
develop teacher expertise; 

• Aligning district policies for curricu- 
lum, professional development, 
resource allocation, and teacher 
evaluation with the vision of re- 
formed practice and the strategies for 
improvement; and 

• Promoting supportive state policies. 
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The leaders of MISE believed that a 
comprehensive approach was needed to 
stimulate, support, and sustain the 
inquiry-centered classroom practice they 
sought. New curriculum materials and 
professional development were essential, 
but school and district policies governing 
curriculum, assessment, professional 
development, resource allocation, per- 
sonnel evaluation, and accountability 
also had to be aligned with the vision of 
high-quality science teaching if the 
reforms were to take root in classrooms 
and be sustained. 

The Partnership’s 
Professional Development 

Professional development is the 
central driver in the MISE theory of 
action. MISE and the Partnership have 
designed professional development with 
four key objectives in mind: effective 
implementation of newly adopted 
curriculum modules in science, the use 
of inquiry as a primary instructional 
strategy in science teaching, the use of 
performance assessment as a source of 
formative feedback for instruction, and 
the development of teacher leadership. 
Over time, the Partnership has pursued 
these objectives through a variety of 
complementary professional develop- 
ment activities. In the first two years of 
MISE's partnerships with the four 
districts, priority was placed on the 
review and selection of new standards- 
based science curriculum modules for 
use in grades K-8. Although MISE 
supported related district professional 
development programs, it quickly be- 
came apparent that these local efforts 
were not robust enough to stimulate and 
support the changes in practice that 
MISE envisioned. 

It was at that point that MISE and 
the four districts formed the Partnership 
and adopted a common professional 
development strategy. The Partnership 



began its professional development work 
in 1994 with the selection of teams of 
three-to-four "Leader Teachers" from 
each school serving grades K-8. These 
teams attended three-week summer 
institutes and five-to-seven follow-up 
sessions each year for three consecutive 
years. Known as the Leader Teacher 
Institute (LTI), these experiences were 
not only intended to enhance partici- 
pants' knowledge of science content and 
their skills in inquiry-centered teaching, 
but to prepare them to be leaders in their 
schools and districts. The LTI was de- 
signed and delivered by instructional 
teams that included specialists in sci- 
ence, pedagogy, and leadership. The 
participating teacher teams were ex- 
pected to promote and support the 
implementation of the new curricular 
units, the use of inquiry-centered in- 
struction, and the introduction of perfor- 
mance assessment. 

In 1996, a complementary strategy 
was introduced. With the support of an 
ESC grant from NSE, the Partnership 
extended access to professional develop- 
ment in science and mathematics to all 
elementary and middle school teachers 
through multi-day summer programs 
known as Peer Teacher Workshops 
(PTWs). Typically, a PTW focused on a 
single multi-week science unit, address- 
ing curriculum content, classroom and 
unit management, pedagogy, assess- 
ment, and responses to common student 
misunderstandings and learning prob- 
lems. The selection of PTWs offered each 
summer was determined by the needs 
and interests of teachers and the adop- 
tion of new science units by the four 
districts. PTWs were provided in math- 
ematics, technology, and assessment as 
well as science. Teachers, including the 
Leader Teachers, could attend multiple 
sessions during the summer, and they 
could participate in PTWs year after 
year. The PTWs were planned and led 
by instructional teams composed of 
district staff, accomplished teachers. 



7 



Getting It Right: The MISE Approach to Professionai Deveiopment 



MISE staff, and external consultants 
who helped to ensure quality of the 
design and implementation. In the first 
two years, the PTWs were jointly 
planned and were offered in central 
locations for teachers from all four 
districts. Based on CERE findings that 
this created a barrier to participation for 
some teachers, the Partnership shifted to 
a district-based strategy for delivering 
the PTWs but the four districts contin- 
ued to collaborate on planning, and 
teachers crossed district lines to attend 
the programs. 

Careful attention to the design of 
learning experiences for adults and the 
continuous improvement of these de- 
signs were hallmarks of the PTW pro- 
gram. Each spring, MISE held a two-day 
design retreat for the instructional teams 
responsible for designing and delivering 
the PTWs in all four districts. Team 
members were introduced to the expec- 
tations and standards of the 
Partnership's curriculum-based ap- 
proach to professional development, 
provided with assistance as they de- 
signed the next round of summer insti- 
tutes, and introduced to a professional 
learning community focused on the 
development and sharing of knowledge 
about effective strategies for engaging 
teachers and supporting their learning. 
This strategy of assisted design informed 
by experience contributed significantly 
to the quality of the PTWs. 

A third strategy focused on princi- 
pals. In the initial years, MISE provided 
professional development for the princi- 
pals around the science standards, the 
use of inquiry, and how to work with the 
Leader Teachers to improve practice. In 
later years, the Partnership offered a 
series of institutes for principals that 
focused on what to look for in the class- 
room, revision of observation instru- 
ments, and how to assist teachers with 
the improvement of their practice. 



The provision of on-site follow-up 
support, the fourth strategy, was an 
essential feature of both the LTI and 
PTW programs. The MISE and district 
supervisory staffs provided support for 
the Leader Teachers. The Partnership 
took a four-fold approach to follow-up 
for the PTWs: supporting formal meet- 
ings of PTW participants during the 
school year, preparing district science 
supervisors to support reformed class- 
room practice, organizing on-site sup- 
port from the Leader Teachers, and 
preparing principals to know what to 
look for in classrooms. 

Einally, the Partnership supported 
teacher networks. Teachers were re- 
cruited to work on curriculum frame- 
works, performance assessments for use 
in the science modules, and the adminis- 
tration of performance assessments in 
several grades. This work was typically 
done by study groups of teachers who 
worked within, and across, the four 
districts to develop these new tools for 
the improvement of instruction. Sup- 
ported by MISE staff and other consult- 
ants, these study groups and networks 
provided rich professional development 
for many teachers in the four districts. 

Below, we take a more detailed look 
at the first three of these strategies. 

The Leader Teacher Institute 

The LTI was designed to prepare 
teams of Leader Teachers to support the 
spread of reformed science teaching in 
their schools. The objectives were to 
deepen their knowledge of science, 
enhance their skills in inquiry-centered 
pedagogy, and prepare them to lead the 
improvement of science teaching in their 
schools and districts. The Leader Teach- 
ers were expected to: 

• Embrace and spread the use of 

inquiry-centered teaching; 
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• Increase their knowledge of science; 

• Implement inquiry-centered instruc- 
tion in their classrooms; 

• Develop the "habits of mind" of 
science learners; 

• Design and use assessments that 
informed instruction; 

• Integrate science, mathematics, and 
technology with other disciplines; 

• Create classrooms in which all 
students were fully engaged; 

• Model and demonstrate these prac- 
tices for their colleagues; 

• Assume responsibility for their own 
continuing professional develop- 
ment; and 

• Provide leadership within their 
schools and communities. 

For several years, the LTI was the 
Partnership's core activity. It raised the 
visibility of science in the four districts 
and spread the vision of inquiry-cen- 
tered science instruction. It helped 
transform four district partnerships with 
MISE into a single collaborative effort — 
the Partnership. It stimulated school- 
wide initiatives to improve science 
teaching, and it altered the roles of MISE 
staff as they became change agents and 
providers of technical assistance. 

Peer Teacher Workshops 

In its third year, the Partnership 
received a major grant from NSP's ESC 
program. The primary objective was to 
provide, over a five-year period, 100 
hours of high-quality professional devel- 
opment in science and mathematics to at 
least 80% of the nearly 800 K-8 teachers 
who taught these subjects in the four 



partner districts. The NSE grant enabled 
the Partnership to move beyond the 
preparation of Leader Teachers and 
directly engage all K-8 teachers in the 
four districts. 

Three-to-four-day summer work- 
shops combined with follow-up sessions 
during the school year were first offered 
in the summer of 1996. Participation 
was voluntary but encouraged by the 
districts and all teachers were invited to 
attend. Principals, Leader Teachers, 
science supervisors, and others recruited 
participants. Initially, the workshops 
focused on the science modules adopted 
by the districts and they were organized 
by grade-level clusters. 

In the first summer of the PTWs, 169 
regular classroom and special education 
teachers (87 in New Jersey and 82 in 
Pennsylvania) participated in one of 
three theme-based sessions — Observing 
and Measuring in the Real World 
(grades pre-K-2), About Matter (grades 
3-5), and Environmental Science: Sus- 
taining the Earth (grades 5-8). Each 
summer since, PTWs have been offered 
to teachers in Partnership schools. The 
PTWs have been expanded over time to 
cover mathematics, technology, and 
assessment as well as additional science 
modules. The PTWs broadened the scope 
of the Partnership's work, engaging 
more teachers in the reform of science 
and mathematics education. 

Participation in the PTWs grew 
steadily during the first few years of the 
ESC project. Table 3 shows the numbers 
of teachers participating each summer 
from 1996 to 2002. Erom 1996 to 2002, 
more than 80% of the eligible teachers 
attended two or more of the summer 
workshops, and over 60% attended 
three or more. Each PTW involved 30 
hours of professional development 
during the summer and follow-up 
sessions during the school year, repre- 
senting a significant investment of 
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Table 3 


. Number of Peer Teacher Workshops 
and Enrollment, 1996 to 2002 




Year 


Number of 
PTWs 


Math 

PTWs 


Math and 
Science 
PTWs 


Science 

PTWs 


PTW 

Enrollment 


# of Individuals 


Summer 1996 


6 


0 


0 


6 


169 


169 


Summer 1997 


8 


2 


0 


6 


195 


195 


Summer 1998 


22 


8 


1 


13 


506 


287 


Summer 1999 


36 


9 


1 


26 


525 


386 


Summer 2000 


32 


9 


2 


21 


667 


490 


Summer 2001 


37 


12 


1 


24 


536 


394 


Summer 2002 


25 


0 


0 


25 


581 


361 


Total 


166 


40 


5 


121 


3,179 


2,282 



teacher time in professional develop- 
ment. 

Instructional Teams. The PTWs were 
designed by teams of three or four 
workshop instructors including a science 
or mathematics content specialist, an 
experienced professional developer, and 
one or two accomplished classroom 
teachers. The inclusion of accomplished 
teachers from the partner districts was a 
deliberate effort to ground the PTWs in 
the realities of classroom practice in the 
participants' context. It also was in- 
tended to build the capacity of the 
districts by developing cadres of teachers 
who could confidently and expertly 
design and lead professional develop- 
ment and serve as peer mentors and 
coaches. The teams were brought to a 
two-and-a-half-day retreat in Princeton, 
NJ each spring. Designed by MISE staff, 
the retreat was intended to provide the 
instructional team members with a 
shared vision of high-quality profes- 
sional development, time to design their 
PTW, and assistance with design. The 
objectives for the retreats were to: 

• Establish good working relationships 
among instructional team members; 

• Make the roles of the team members 
explicit; 



• Assess the strengths and needs of 
individual team members; 

• Review the elements of effective 
professional development and apply 
them in designing the content and 
format of PTWs; 

• Establish expectations and guide- 
lines for designing workshop ses- 
sions, ordering workshop supplies, 
scheduling team planning meetings, 
producing workshop materials, 
documenting workshop planning 
and facilitation, and tracking partici- 
pant attendance; and 

• Begin the planning of the PTWs for 
the summer. 

At the retreats, district and MISE 
staff demonstrated good professional 
development. Participants engaged in 
hands-on investigations and tried out 
strategies for reviewing student work. 
They reviewed the standards for profes- 
sional development and discussed 
situations that might arise in the course 
of a PTW. Ample time was always 
allowed for teams to meet to plan their 
summer workshops. 

The instructional teams met through- 
out the spring to complete their designs 
and assemble materials; MISE staff met 
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with them to assist as needed and to 
review the science content with the team 
members. 

From the second year on, the instruc- 
tional teams had the designs, materials, 
and evaluation data from the previous 
years' PTWs available to guide their 
work. As a result, the designs were 
elaborated and improved over time. In 
interviews with CPRE evaluators, the 
instructional team members gave glow- 
ing accounts of the retreats. They re- 
ported that they had learned a great 
deal about science and how to teach it, 
as well as acquiring knowledge about 
how to design and guide learning expe- 
riences for adults. 

During the PTWs, the instructional 
teams modeled the pedagogy they were 
trying to get teachers to use, and partici- 
pants conducted investigations, worked 
in cooperative learning groups, analyzed 
instructional activities against standards, 
and reflected on their current practices 
and what they were learning. Teachers 
participated in guided inquiry them- 
selves, explored the science content of 
the modules, observed demonstrations to 
illustrate key concepts, designed chal- 
lenging instructional activities for their 



students, considered alternative ways of 
assessing student understanding, and 
discussed student engagement, logistics, 
and classroom organization. 

The Quality of the Instructional 
Teams. Did the PTW participants think 
that the instructional teams possessed 
the necessary knowledge and skills? Did 
they find them helpful? Participants 
completed evaluation surveys at the end 
of each PTW. Year after year, the vast 
majority reported being highly satisfied 
with the PTWs, and their responses to 
questions about the PTW instructional 
teams were overwhelmingly favorable. 
Teachers gave their highest ratings to the 
instructional teams' knowledge of sci- 
ence/ mathematics content and their 
ability to model inquiry-centered instruc- 
tion. 

Figure 1 presents aggregated partici- 
pant responses to some of the key ques- 
tions about the instructional team that 
led their sessions. Ninety-six percent of 
the participants said the instructional 
team's knowledge of science or math- 
ematics instruction was either very or 
extremely effective. In addition, almost 
90% of the participants felt that the 



Figure 1 . Percentage of PTW Participants Who Found Selected 
Aspects of their Instructional Team's Delivery Very Effective or 
Extremely Effective, 1996 to 2000 
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instructional teams were very or ex- 
tremely effective in their ability to model 
inquiry-centered instruction, in their 
ability to respond to participants' ques- 
tions and feedback, and in their skill in 
instructing adult learners. 

Each year, some of the instructional 
team members were conducting work- 
shops for the first time, and despite 
MlSE's efforts to provide them with 
guidance and support (through the 
design retreat and the inclusion of 
content experts on the instructional 
teams), there was always some uneven- 
ness in the delivery of PTWs. However, 
the overall impression of CERE evalua- 
tors was that only a modicum of quality 
was sacrificed by recruiting accom- 
plished teachers to serve on the teams, 
and the strategy helped build greater 
district capacity in the long run. 

Based on the sessions observed by 
CERE staff and follow-up interviews 
with participants, most PTW leaders 
were perceived by participants to be 
knowledgeable and skilled. Effective 
instructional team members stimulated 
participants' interest in science and 
helped them master the content of the 
curriculum. A typical participant com- 
mented: 

This has been perhaps one of the best 
workshops I have attended because of 
the facilitator's preparedness, style, and 
knowledge of the classroom and 
program. I feel much better prepared to 
implement the program, to assess it, 
and to share my knowledge and 
techniques with my peers. 

Another respondent said: 

The instructional team did a wonderful 
job. This is the second time that I've 
been in a workshop with [instructor]. 
She makes us all feel that we are 
physicists. Certainly, all aspects [of the 
workshop] can be used, but I found the 
science content and inquiry-centered 



instructional techniques to be most 
helpful for me. Being able to revisit 
content in this new light helped to 
refresh my memory and made clearer 
for me the great value of using inquiry- 
centered methods. 

When asked what they intended to 
use in their classrooms, participating 
teachers mentioned the inquiry ap- 
proach, better questioning techniques, 
specific content knowledge, and new 
assessment tools. These were the out- 
comes the Partnership was seeking. 
While intentions do not always predict 
changes in practice, the data from 
follow-up surveys and interviews sug- 
gest that most of the participants put 
their newly acquired knowledge and 
skills to use in their classrooms. 

CERE evaluators also interviewed 
instructional team teachers several times 
in order to obtain their assessment of the 
PTW experience. Did they feel prepared 
to design and lead a workshop? Did 
they receive the support they needed? 
Did they feel respected by other team 
members and the participants? Did they 
believe that the workshops were chang- 
ing classroom practice? The answers to 
these and related questions were univer- 
sally positive. The strategies of focusing 
on curriculum units, co-constructing the 
designs, and using accomplished practi- 
tioners to lead them were perceived to be 
the keys to their success. The teachers 
reported that they were clear about the 
skills they brought to the instructional 
team and that they were satisfied with 
their role on the team. As one teacher 
remarked: 

I was part of the basic decision-making 
process. We were given a basic 
framework, but we were allowed to 
shape it from the ground up within our 
discipline, including what our themes 
would be and what activities we might 
use. We all brought to the table what we 
had or could find. 
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All of the teachers said that they 
were initially uncomfortable as instruc- 
tors but quickly grew into the role. They 
all felt that the experience had deepened 
their understanding of science and 
affected their classroom practice, and 
many indicated that it had empowered 
them and altered their aspirations as 
professionals and the opportunities for 
influence within their districts. 

Principals’ Institutes 

The implementation of instructional 
reforms requires support from school 
principals. In study after study, research- 
ers examining the factors affecting the 
implementation of reforms have con- 
firmed this finding (Elmore, 2000; 
Newmann & Associates, 1996; Spillane, 
Halverson, & Diamond, 2001). Strong 
principals provide their staffs with vision 
and focus, creating coherence within 
their schools. They support the efforts of 
teachers to improve their practice, 
building strong professional communi- 
ties that focus on results and promote 
collaboration, and providing assistance 
to teachers who need it. They allocate 
resources, including time to support the 
instructional priorities of their schools, 
and buffer their staffs from distractions. 
In sum, principals play a critical role in 
improving instruction. 

Although MISE recognized the 
important role played by principals, it 
focused on strengthening the science 
curriculum, developing Leader Teachers, 
and providing professional development 
for teachers. Initially, MISE introduced 
principals to the science standards and 
their vision of reformed practice, and 
worked with them on the effective use of 
the Leader Teachers in their schools. But 
during the middle years of the Partner- 
ship, 1997 to 2000, MISE focused more 
on the engagement of central office 
personnel and teachers than it did on 
principals. The assumptions were that 
the superintendents, central office staff, 
and Leader Teachers would win over the 



principals, gain their support for the 
Partnership's work, and provide them 
with whatever preparation they needed 
to support reforms in science teaching. 

Peedback from the field soon indi- 
cated that the principals needed more 
preparation to support the reforms in 
science teaching. The CPRE evaluation 
of the work of the Leader Teachers 
revealed that some did not receive 
support from their principals. Teachers 
who attended the PTWs told CPRE 
evaluators that the priorities and atti- 
tudes of their principals determined the 
degree to which they could use inquiry 
methods in science. Also, teachers re- 
sponding to the HRI survey reported 
considerable variation in the support 
being provided by their principals. 
District leaders also expressed concern 
that turnover meant that there were 
many new principals who had only 
vague notions of the instructional vision 
of MISE and the Partnership. 

The results of the HRI administrator 
surveys provided a somewhat more 
positive picture than the other data 
sources perhaps because the principals 
felt some pressure to indicate that they 
were involved. Nevertheless, about a 
quarter of the principals reported some- 
thing less than active involvement in the 
Partnership's work on the 2001 survey 
(see Table 4). 

As a consequence, during 2000-2001, 
MISE convened a representative group 
of principals from the four districts to 
discuss what should be done to prepare 
principals to be effective instructional 
leaders in science. The result was a 
professional development program for 
principals that focused on what to look 
for in science classrooms, distributed 
leadership and how to make effective 
use of accomplished teachers, and 
supporting teacher learning on the job. 
The first two-day institute in June 2001 
was attended by 41 principals and some 
central office staff. Three other sessions 
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Table 4. The Principals^ Perspective of the Partnership, 
Results of the 2001 HRI Survey 



Item 


Not at all 


Responses 

Somewhat 


To a great extent 


How familiar are you 


6.5% 


9.7% 


83.9% 


with the LSC project? 








To what extent have 


13.0% 


9.7% 


77.4% 


you been involved? 









followed, and were also well attended. 
These Partnership Principals' Institutes 
were planned by a committee of princi- 
pals from the four districts and MISE. 

Principals' responses to the institutes 
were overwhelmingly positive. Follow- 
up interviews revealed a demand for 
more experiences focused on instruc- 
tional practice and an appreciation of 
opportunities to interact with peers from 
other districts. By bringing the principals 
together, the Partnership helped develop 
a community of practitioners who could 
learn from one another. Participation in 
the institutes was higher than expected, 
and the principals requested additional 
sessions. One outgrowth of these ses- 
sions was the revision of local teacher 
observation instruments to make them 
more consistent with the Partnership's 
vision of good instruction. 

Did the Partnership’s 
Professional 
Development Meet the 
Standards? 

Did the learning opportunities for 
teachers designed by the Partnership 
meet the consensus standards for high- 
quality professional development that 
we described earlier? Overall, CPRE's 
assessment is that the Partnership's 
model of active and deep engagement 
with curriculum content and pedagogy 
combined with follow-up during the 
school year and in the workplace was 



consistent with the consensus view of 
quality and with the standards adopted 
by the National Staff Development 
Council (2001; Killion et al., 2003), the 
American Federation of Teachers (1999), 
and the National Institute for Science 
Education (Foucks-Horsley, Stiles, & 
Hewson, 1996). The MISE staff under- 
stood the standards and applied them, 
and they were committed to improving 
the quality of their work. They actively 
sought and used feedback on their 
processes and designs from participants, 
district staff, and CPRE evaluators. A 
cyclical process of design, implementa- 
tion, feedback, and redesign was central 
to their work. Here, we take a look at 
how well the Partnership's work mea- 
sured up to the 12 standards presented 
earlier. 

The first standard states that profes- 
sional development should be solidly 
grounded in research and clinical 
knowledge of teaching and learning. The 
science content of the LTI and PTW 
programs was based on the national and 
state science education standards, which 
are the best available syntheses of re- 
search and clinical knowledge in science 
education (National Research Council, 
1996; Olson & Loucks-Horsley, 2000). 
These programs focused on key concepts 
in the physical, biological, and earth 
sciences included in the K-8 curriculum 
and on the use of inquiry, which is 
treated as an essential aspect of science 
content in the national standards. The 
inclusion of expert teachers, district 
supervisors, and pedagogical experts on 
the LTI and PTW instructional teams 
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ensured a solid grounding in clinical 
knowledge of teaching and learning. 

The second standard states that 
professional development should be 
aligned with the curriculum that teach- 
ers are expected to teach, and the PTWs 
certainly were well aligned. Approxi- 
mately two-thirds of the PTWs were 
designed to help teachers implement 
specific science curriculum modules. The 
remaining workshops focused on writ- 
ing in science, mathematics, integrating 
technology, and student assessment. 

The third standard recommends 
focusing on student learning in a par- 
ticular context. Both the LTI and PTW 
experiences focused heavily on develop- 
ing teachers' understanding of the core 
science concepts covered in the local 
curricula and on student understanding 
of them. The PTWs were designed 
around specific curricular units being 
offered to particular groups of students. 
Considerable attention was directed to 
how students might respond to these 
activities and what concepts or proce- 
dures they might misunderstand, how to 
engage students in inquiry, and how to 
assess student understanding. Since the 
instructional teams included teachers 
who had taught the units to similar 
groups of students, they were able to 
help participants make these connec- 
tions to their classrooms. The follow-up 
sessions sometimes involved looking at 
student work and almost always in- 
volved discussion of real teaching and 
learning problems encountered in Part- 
nership classrooms. 

The fourth standard recommends 
modeling of good practice. MISE staff 
certainly encouraged this in the design 
retreats. They modeled good practice 
themselves, and helped others do so by 
coaching them. Both LTI leaders and 
PTW instructional team members mod- 
eled the pedagogy they were trying to 
get teachers to use. Participants con- 



ducted investigations, worked in coop- 
erative learning groups, received feed- 
back from instructors and other teachers 
about their instruction, and reflected on 
what they were learning. Leader Teach- 
ers were often asked to model the use of 
inquiry for others who were having 
difficulty implementing it in their classes. 
MISE professional development was 
designed to enable teachers to demon- 
strate and share their knowledge so it 
could spread throughout their schools. 

The fifth standard emphasizes the 
importance of active learning opportuni- 
ties for teachers including practice, 
feedback, and reflection. Both the LTI 
and the PTWs provided low-risk, col- 
laborative environments and structured 
opportunities for practice, peer feedback, 
and time for reflection. Participating 
teachers conducted experiments, ob- 
served and critiqued demonstrations, 
planned and carried out inquiry-based 
activities, and reflected on their experi- 
ences. The follow-up sessions provided 
rich opportunities for reflection as 
teachers could share accounts of their 
efforts to use the modules and strategies, 
discuss their students' responses and the 
problems they encountered, and exam- 
ine student work. 

The sixth standard — the transpar- 
ency of the limitations of the evidence 
supporting the desired practice — will 
be discussed at the end of this section. 

The seventh and eighth standards 
address the intensity, duration, and 
persistence of the experiences and the 
presence of on-site support. The LTI and 
PTWs clearly offered teachers opportuni- 
ties to become deeply immersed in 
science content and pedagogical content 
knowledge about science. The LTI ran 
for three weeks each summer, and 
included five-to-seven follow-up days. 
The PTW participants were together for 
three or four days and received two half- 
days of follow-up during the year. And, 
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they received on-site support from 
science supervisors and Leader Teachers 
who were familiar with the curriculum 
content. Moreover, these opportunities 
for professional development were 
sustained over time, giving teachers time 
to integrate inquiry into their practice 
gradually, to reflect on their experience, 
and to deepen their pedagogical knowl- 
edge. The typical teacher attended more 
than three summer PTWs during the 
first seven years they were offered. 

The ninth standard requires atten- 
tion to teachers' knowledge of subject 
matter and their pedagogical content 
knowledge. The Partnership's intent was 
to build and deepen teachers' content 
knowledge and strengthen their peda- 
gogical skills. The LTl concentrated on 
teacher content knowledge and teachers' 
mastery of critical concepts in the physi- 
cal, biological, and earth sciences. Sci- 
ence content was taught through lec- 
ture, demonstration, hands-on activities, 
and reading. The PTWs focused more 
narrowly on the specific content and 
concepts in the curriculum modules as 
well as pedagogical techniques and 
assessment. Less time was devoted to 
content mastery in these four-day expe- 
riences as management of the unit, 
logistics, pedagogy, and assessment also 
had to be addressed. The emphasis 
tended to be placed on the content that 
students might have trouble with, and 
on ways of presenting or explaining that 
content. 

The tenth standard requires utiliza- 
tion of teacher expertise and leadership. 
Both the LTI and the PTW programs 
involved teachers and administrators in 
planning, design, and delivery. Some 
teachers and local curriculum experts 
joined MISE staff to plan the LTI, and, of 
course, the central purpose was to 
prepare teachers from each school to be 
Leader Teachers. As we have recounted 
elsewhere, many of the Leader Teachers 



played significant roles in the develop- 
ment of curriculum and assessment in 
their districts as well as serving as men- 
tors and coaches.® A different, and more 
inclusive, strategy was used for the 
PTWs. Teachers who had been success- 
ful in the LTI and other accomplished 
practitioners were asked to be instruc- 
tional leaders for the PTWs. Leader 
Teachers were involved in district profes- 
sional development planning and in the 
design and delivery of the PTWs. Mem- 
bers of the PTW instructional teams 
gained considerable skill and confidence 
in working with their colleagues. They 
received guidance from MISE and their 
team members, and the teams debriefed 
on a daily basis and reflected on how the 
activities they had planned had worked 
and how well they had carried them 
out. This approach to learning to work 
with adults proved quite successful, and 
over the years, this cadre of teachers 
expanded until each district had a 
significant number of teachers able to 
lead professional development. 

The eleventh standard requires 
collaboration among teachers within 
and across schools. The Partnership used 
a variety of strategies to foster collabora- 
tion among teachers both within and 
across schools — training Leader 
Teacher teams, encouraging grade-level 
teams to attend PTWs together, conduct- 
ing school meetings, supporting study 
groups for assessment, and creating 
electronic networks. The high level of 
participation in the PTWs over time is 
just one measure of their success. The 
Leader Teacher teams were expected to 
function as units in their schools, to 
build communities of practice around 
science, and to encourage teachers to 
sign up for the summer workshops. The 
success of these teams varied, depending 
on the status of the Leader Teachers in 
the school, the support provided by their 



®See CPRE (1999) and CPRE (2000). 
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principals, and whether there was time 
for teachers to meet. In most of the 
schools, however, they were successful 
in getting most faculty engaged in 
improving science instruction. Dozens of 
teachers were also involved in study 
groups that worked on the development 
of performance assessments to be used 
with the science modules. These teachers 
also became members of professional 
communities that cut across school and 
district boundaries. Many of these 
teachers later were involved in the 
Partnership-wide adaptation and ad- 
ministration of TIMSS (Third Interna- 
tional Mathematics and Science Study) 
performance assessments in selected 
grades. They worked together on revis- 
ing, scoring, and administering these 
performance tasks, and also on inter- 
preting the results. 

Finally, consistent with the last 
standard, teachers who participated in 
the LTI and the PTWs consistently and 
almost universally reported being treated 
as professionals. At the instructional 
team retreats, facilitators modeled, and 
then made explicit, the relationships that 
instructors should establish with adult 
learners. Respect and use of prior knowl- 
edge were central themes. In the PTWs, 
instructors drew upon the knowledge 
and experience of the participants to 
discuss management of inquiry in the 
classroom, common logistical problems 
and their solutions, and common con- 
ceptual misunderstandings and alterna- 
tive ways to explain key concepts. The 
effects of this emphasis were apparent in 
follow-up evaluations: teachers consis- 
tently reported that they appreciated 
being treated as professionals. 

In sum, the professional development 
offered by MISE and its partners satis- 
fied most of the standards we described 
earlier in this report. In fact, most of 
these standards were articulated as 
principles of design at the PTW retreat. 
Perhaps the only standard that the 



Partnership failed to meet consistently 
was the sixth one: making the evidence 
supporting their vision of good practice 
accessible, transparent, and subject to 
critique. MISE staff were strong advo- 
cates of inquiry-centered practice and 
did not always address its limitations. 
Inquiry-centered pedagogy and the 
science modules were generally pre- 
sented as "best practice" without much 
discussion of the strength of the evidence 
supporting these claims. Like most 
reformers, they appealed to teachers on 
both philosophical and practical 
grounds, and drew the strongest support 
for their advocacy from professional 
organizations rather than from empirical 
evidence. 

Building District and 
School Capacity 

Did the Partnership's approach to 
professional development build district 
capacity to stimulate and support con- 
tinuous improvements in teaching? Both 
the LTI and PTW programs were de- 
signed to be capacity-building initiatives. 
Analysis of interviews conducted by 
CPRE evaluators with Leader Teachers, 
principals, instructional team members, 
and central office staff reveals persuasive 
evidence that significant changes oc- 
curred in the capacity of all four dis- 
tricts. District leaders consistently ex- 
pressed confidence that their staffs could 
plan and deliver the high-quality profes- 
sional development provided under the 
auspices of the Partnership, and in all 
four districts, the model of instructional 
support used in science was extended to 
other subject areas. The situation in the 
schools was somewhat more mixed, but 
most principals and Leader Teachers felt 
that their schools had gained the capac- 
ity to sustain reforms in science teaching 
and had developed strategies for intro- 
ducing new teachers to their concepts of 
good practice. They, too, reported that 
the Partnership experience had provided 
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them with the attitudes, know-how, and 
tools needed to make improvements in 
other areas of teaching. 

CPRE collected substantial evidence 
from interviews and observations that 
Leader Teachers took their responsibili- 
ties to provide instructional leadership in 
their schools and districts seriously. 
Forty-two Leader Teachers were inter- 
viewed during the 1997-1998 school 
year, and a second sample of 38 were 
interviewed in the 1999-2000 school 
year. Based on the analysis of these data. 
Leader Teachers who took on responsi- 
bilities in their schools roughly fell into 
five categories: serving as on-request 
resources, providing individual outreach 
to teachers, providing individual out- 
reach school-wide, providing team 
outreach school-wide, and serving 
district-wide needs. Both sets of inter- 
views found considerable variation in 
the roles played by Leader Teachers 
across schools, and the second set of 
interviews revealed a slight decline in 
Leader Teacher activity. 

On-request Resources. Virtually all 
the Leader Teachers said that they 
regularly responded to requests for 
information from other teachers and 
served as on-request resources in their 
schools. They assisted individual teach- 
ers with science-related questions, 
explaining key concepts, sharing lesson 
plans, and helping to set up demonstra- 
tions or investigations. For most Leader 
Teachers, this was just one dimension of 
their role, but some did not feel comfort- 
able going beyond responding to such 
requests. Some felt they should not be in 
other teachers' classrooms and worried 
that other teachers would resent more 
proactive roles. In either case, they were 
uncomfortable taking much initiative to 
share their knowledge. 

Individual Outreach to Teachers. 
Many of the Leader Teachers reported 
that they worked on an individual basis 



with other teachers. They referred to this 
work as coaching, in which they worked 
with another teacher over time to help 
them teach one of the science or math- 
ematics modules, to design or modify 
assessment tasks that were more authen- 
tic or more closely aligned with the unit, 
or to develop curriculum. Frequently, 
these other teachers were grade-level 
partners, or, in a few cases, student 
teachers. 

Individual Outreach School-wide. 
Some Leader Teachers reported organiz- 
ing or implementing activities for groups 
of teachers or for the whole school. One 
Leader Teacher said, "My role is building 
strength at my grade level in a coopera- 
tive vein." Leader Teachers from several 
schools reported collaborating on cur- 
riculum with groups of teachers who 
had participated in the PTWs. A special 
education Leader Teacher described how 
she facilitated a workshop on inclusion 
and inquiry-centered science for her 
school's staff. Another Leader Teacher 
explained how she assumed responsibil- 
ity for revamping the school's science 
curriculum. Several Leader Teachers 
mentioned how they coached groups of 
teachers at their grade level or provided 
ongoing support to graduates of PTWs. 

Team Outreach School-wide. Leader 
Teachers in about half of the Partnership 
schools worked as teams to provide 
activities for their school or community. 
These activities varied, but were distinc- 
tive because the Leader Teachers worked 
collaboratively in teams, and not just as 
individuals. In several cases, the collabo- 
ration involved the design of workshops 
for teachers. Other examples included 
organization of science fairs or science 
nights for students and parents, and 
mathematics and science career days for 
students. In another case. Leader Teach- 
ers worked together to develop a school- 
wide structure for developing and 
sharing lesson plans. In a few cases. 
Leader Teachers developed curriculum 
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units around themes (the rainforest in 
one school, a woodland habitat in 
another) that they used to model reform 
strategies for other teachers. These 
strategies included eliciting student 
questions, using assessment to gauge 
prior knowledge, inquiry-centered 
activities, and linking curricula to stan- 
dards. One Leader Teacher said, "In the 
whole school, there has been tremen- 
dous growth. As a group of Leader 
Teachers, we did it [provided support] as 
necessary. We all did different things to 
help within the building. It's very infor- 
mal, but we all get around." 

District-wide Influence. A number of 
Leader Teachers were involved at the 
district level. Several Leader Teachers led 
district in-service days. Leader Teachers 
also assumed an increasingly prominent 
role in planning and leading PTWs in 
their districts. Leader Teachers in each 
district served on science and mathemat- 
ics curriculum and frameworks commit- 
tees. Leader Teachers were an integral 
part of their districts' representation on 
MISE's Advisory Committee, which 
brings together leadership groups from 
the four partner districts to discuss 
strategic issues and formulating policies 
and other reforms. A few Leader Teach- 
ers mentioned that they represented 
their districts in speaking at state science 
conferences. 

School Principals and 
Leader Teachers 

Almost all of the principals reported 
that they supported the work of the 
Partnership and the Leader Teachers in 
their schools. Several principals met 
regularly with their Leader Teachers, as 
one principal described, "to foster the 
Merck initiatives throughout the school." 
An elementary principal said, "The 
Leader Teachers have had a presence in 
the building. They are role models for 
others to emulate." One principal orga- 



nized the school schedule so that Leader 
Teachers could go to other teachers' 
classrooms to support their science 
instruction. Several principals from 
different districts mentioned that schools 
and districts need to better define the 
roles of Leader Teachers. 

Principal support was the most 
powerful predictor of reform-based 
teaching practice. Teacher assessments 
of the amount of support provided by 
their principals were statistically associ- 
ated with reform-based teaching prac- 
tice (CPRE, 1999). Teachers in schools 
with supportive principals were far 
more likely to use inquiry-centered 
practices than teachers in schools where 
the school leader was not supportive. 

The importance of principals and the 
variation in their support also was 
revealed in interviews with Leader 
Teachers. Leader Teachers from several 
schools across the Partnership described 
receiving outstanding support from their 
principals. "Our principal is 150% 
behind the Merck initiative," said one 
Leader Teacher. A principal in another 
school held monthly meetings with her 
Leader Teachers and involved them in 
school decision-making about science 
and mathematics. A Leader Teacher 
from this school commented, "She treats 
us as leaders. She looks for needs in the 
building and.. .uses us as leaders in the 
school." Another Leader Teacher de- 
scribed how her principal "meets with 
us to discuss science and mathematics 
issues before they are brought to the rest 
of the faculty. We are appreciative that 
she asks for our input first. She respects 
us for our efforts with Merck." 

Other Leader Teachers described 
indifferent administrative support in 
their schools. One Leader Teacher said, 
"It is passive support. They are support- 
ive but not involved." Another Leader 
Teacher felt that the continual change in 
the administrative staff made stable 
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support of high-quality instruction 
impossible. "It is not intentional," she 
said. "But things are so vague, you just 
don't know from one day to the next, 
and these things play against the initia- 
tives such as Merck." In several other 
cases. Leader Teachers felt there was no 
place for teacher leadership in their 
schools, and that authority rested with 
the administrative staff, not the faculty. 

Another Capacity-building 
Strategy 

The inclusion of accomplished teach- 
ers on PTW instructional teams also 
contributed to the capacity of the four 
districts. Table 5 shows the general 
make-up of the instructional teams from 
1996 to 2001. The composition of these 
instructional teams evolved over time; in 
the first few years, the teams drew 
heavily on MISE staff and external 
consultants, but also included some 
Leader Teachers. The external consult- 
ants included individuals from national 
curriculum development and technical 
assistance organizations, other science 
organizations, local and regional univer- 
sities, and other school districts. By 2001, 
the team members were predominantly 
recruited from the ranks of teachers and 
central office staff from the four partner 
districts. Thirty-five of the 39 instruc- 
tional team members offering science 
PTWs in the summer of 2001 were 
district staff including 6 from local high 
schools. This compositional change 
reflects MISE's efforts to build district 
capacity to sustain the professional 
development. Although MISE was 



initially the force behind the professional 
development, over time its role was 
reduced as Leader Teachers gained 
confidence and took charge and the 
districts took on increased responsibility 
for the PTWs. 

Interviews with members of instruc- 
tional teams also indicated that they 
were assuming leadership positions in 
their schools and districts. Some had 
already been playing such roles as they 
had been Leader Teachers, but others 
had not. They reported that being mem- 
bers of instructional teams gave them a 
new status among their peers as well as 
increased confidence in their knowledge 
and skills. The experience of designing 
and leading professional development 
sessions prepared them to lead profes- 
sional development in their schools. It 
also linked them to networks of school 
and district leaders who were involved 
in planning the PTWs for their districts 
and a larger network of educators who 
were working on these tasks across the 
Partnership. 

One instructional team member who 
had not been a Leader Teacher described 
how her role had been transformed: 

I have always liked teaching science, 
and occasionally when I did something 
that involved parents, my principal took 
notice. But since I have been doing the 
PTWs, he asks my advice all of the time, 
gave me a student teacher for the first 
time, and even asked me to do a 
workshop on science for an in-service 
day. 



Table 5. Composition of Instructional Teams for 
Science Workshops, 1996 to 2001 



Background 


1996 


1997 


1998 


1999 


2000 


2001 


Consultants 


4 


5 


6 


11 


20 


4 


District Staff 


7 


14 


24 


43 


40 


35 


Totai 


11 


19 


30 


54 


60 


39 



Source: CPRE, 2001 
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Another member who had been a 
Leader Teacher said her colleagues now 
viewed her differently: 

When I was in the Leader Teacher 
program, I think that some of the other 
teachers felt that I was trying to make 
myself important and resented me or 
just ignored me. But working in the 
PTWs seems to have changed how they 
look at me. It took a while, but they now 
seem to recognize that I know 
something about science teaching and 
they are coming to me for help. Two 
have even asked me into their 
classrooms to observe. 

It is clear from both the survey and 
interview data that the instructional 
team members contributed to school and 
district capacity in several ways: provid- 
ing their districts with the capacity to 
sustain the professional development 
independently, serving as advocates for 
high-quality professional development, 
helping build and sustain professional 
learning communities in their schools, 
and providing role models for other 
teachers to take leadership roles. 

The Impact on 
Classroom Practice 

In this section, we examine how the 
professional development offered by the 
Partnership affected the classroom 
practice of participants. To what extent 
was the knowledge that teachers gained 
through these professional development 
experiences translated to classroom 
practice? This section assesses the evi- 
dence of impact on teacher content 
knowledge and then presents findings 
from five separate analyses of the impact 
of Partnership professional development 
on classroom practice conducted by 
CPRE between 1998 and 2002. 



Teacher Content Knowledge 

Strengthening the content knowl- 
edge of teachers was a major goal of the 
Partnership's professional development 
program. However, no objective mea- 
sures of teacher content knowledge were 
available to the district planning teams, 
the instructional teams leading the LTI 
and the PTWs, or the CPRE evaluation 
team. CPRE's evaluation team did have 
access to a proxy measure, the number 
of college science courses taken by the 
teachers. But this is at best a crude 
baseline measure for relatively inexperi- 
enced teachers. It is less useful as an 
indicator of the knowledge of experi- 
enced practitioners. Classroom observa- 
tions provide little help in this regard. 
While observers can identify those with 
very strong or weak command of their 
subject, single observations do not 
provide reliable measures of teachers' 
practice let alone their content knowl- 
edge. So, for the most part, all parties 
had to rely on teachers' self-assessment 
of their science knowledge as reported 
on the workshop surveys. While these 
measures are not robust, they do reveal 
that both LTI and PTW participants 
believed that the professional develop- 
ment strengthened their content knowl- 
edge. 

Participants in the LTI summer 
session were asked in September 1996 to 
evaluate the three-year LTI and indicate 
how their knowledge and skills had 
changed as a result of the experience. 
Pigure 2 summarizes their responses. 

The results suggest that the teachers had 
gained significant knowledge in the 
areas of science, inquiry-centered in- 
struction, assessment, and cooperative 
learning. Increased knowledge of science 
represented the major area of growth 
while increased knowledge of assess- 
ment was the smallest. Teachers' initial 
estimates of their knowledge of inquiry- 
centered instruction and assessment 
techniques were higher than the previ- 
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Figure 2. Change in Leader Teachers^ Knowledge, 
Pre- and Post-Mean LTI Ratings 
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ous year's, suggesting that they per- 
ceived a real change in their capacity. 

Similar questions were posed to the 
PTW participants each year. And each 
year, over 90% of those attending science 
PTWs reported that they had increased 
their knowledge of the subject. This 
general pattern was confirmed in inter- 
views in which teachers were asked to 
give explicit examples of the knowledge 
that they had gained. The teachers also 
reported that the PTWs provided them 
with the pedagogical content knowledge 
they needed to implement the science 
units successfully. The PTW instructional 
teams assumed that teachers would 
need help with the content of new 
curriculum units and generally relied on 
anecdotal information to identify these 
needs. Those who were delivering PTWs 
previously offered used the archival 
binders, which included notes from 
previous trainers about the concepts and 
topics that were challenging for teach- 
ers. The instructional teams also at- 
tempted to address needs of individuals 
through tutoring and coaching within 
the context of the PTWs, but this was 
limited by the time available and the 
needs of the group as a whole. It should 
be noted that participation in the PTWs 



was voluntary, and teachers whose 
content knowledge in science was 
particularly weak may have chosen not 
to participate. 

The attention paid to the content 
knowledge of the participants varied 
somewhat across the PTWs. It seemed to 
CPRE evaluators that in some cases less 
attention was paid to content than 
might have been desirable, given the 
background knowledge of the workshop 
participants. The participants themselves 
obviously did not share this view; they 
felt that content knowledge was ad- 
dressed adequately and reported so on 
the annual CPRE surveys. They felt their 
content knowledge had been deepened, 
while both CPRE and MISE staff felt that 
more attention needed to be given to it. 

In summary, CPRE evaluators agreed 
with most PTW participants who felt 
that they learned a great deal of science 
content through the PTWs, but the 
evidence to support this claim was 
weak. CPRE's conclusion was based on 
our observations of the content discus- 
sions in the PTWs, the amount of time 
devoted to such discussions, the atten- 
tion given to content in the workshop 
materials, and teacher self-reports about 
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what they learned. However, CPRE 
evaluators also concluded that even 
more attention needed to be given to 
content knowledge if the participants 
were to be expected to make effective 
use of the new science curriculum. 

Classroom Practice 

Analysis 1: Teacher Participation 
and Changes in Practice. To examine the 
association between participation in 
Partnership professional development 
and instructional practice, CPRE evalua- 
tors constructed a statistical model to 
test the magnitude and significance of 
this relationship. This model controlled 
for teacher background and school 
characteristics. A sophisticated form of 
regression analysis, hierarchical linear 
modeling, that recognizes that class- 
rooms are nested within schools, was 
used (Bryk & Raudenbush, 1992). 

The sample used for this analysis 
consisted of 334 science teachers. Using 
items from the HRI surveys, CPRE 
evaluators constructed a scale of re- 
formed teaching practice in science 
classes. The items asked teachers how 
frequently they used certain practices, 
such as requiring students to supply 
evidence to support their claims, demon- 
strating a science-related principle or 
phenomenon, and using assessment to 
find what students know before or 
during a curriculum unit. The number of 
college science courses taken by the 
teachers, years of teaching experience, 
and amount of professional development 
were used as independent variables to 
predict teaching practice. The teachers 
had, on average, 2.36 semesters of 
college science courses, and 6-10 years of 
teaching experience. Twenty-eight 
percent of the 334 teachers reported they 
had received no science-related profes- 
sional development during the previous 
year, 33% reported receiving between 1 
and 39 hours, 17% reported receiving 
between 40 and 79 hours, and 22% 
reported receiving 80 or more hours. 



The most striking result of this analy- 
sis was the statistically strong relation- 
ship between the amount of Partnership 
professional development teachers had 
participated in and their use of inquiry- 
centered pedagogy. After adjusting for 
differences in teachers' content back- 
ground, teaching experience, and school 
environment, teachers with 80 hours or 
more of professional development were 
significantly more likely to be using 
reform-based teaching practices than 
teachers who had less than 79 hours of 
professional development. 

Pigure 3 depicts the relationship 
between teaching practice and profes- 
sional development. It presents the 
predicted use of reform-based teaching 
practice for the average teacher (that is, 
a teacher having the sample's average 
content background and years of experi- 
ence) from the average elementary 
school (that is, a school of average size, 
having the average proportion of stu- 
dents eligible for free or reduced-price 
lunch, and having average resources 
and principal support). 

Teachers who had no professional 
development and teachers who had 
between 1 and 39 hours of professional 
development had approximately average 
teaching practice (that is, their scores on 
the scales developed from the survey 
items were at about the mean). Teachers 
who had between 40 and 79 hours of 
professional development were slightly 
above average in terms of their use of 
reform-based teaching practice. Teachers 
who received 80 or more hours of pro- 
fessional development were much more 
likely to have altered their practice. 

A relationship was found between a 
teacher's content familiarity, measured 
by the number of college science courses 
taken, and reform-based teaching prac- 
tice. Each additional semester of college 
science (a proxy for content familiarity) 
was associated with a statistically signifi- 
cant .11 of a standard deviation increase 
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Figure 3. Impact of Professional Development on Teacher-reported 

Science Teaching Practice 




in the model's measure of reform-based 
teaching practice. This suggests that 
content knowledge is an important 
mediator of the use of inquiry-centered 
instruction. 

Analysis 2: Classroom Observations. 
In 1998, CPRE evaluators observed 68 
science classroom lessons. The observers 
did not know how much Partnership- 
sponsored professional development the 
teachers had experienced, and the 
participation of the observed teachers in 
Partnership professional development 
varied widely. Some individuals had 
received no Partnership professional 
development although they might have 
participated in some other science 
professional development activities 
during this period. However, as a group, 
they had limited opportunities as the 
Partnership was virtually the only 
provider of professional development in 
science in the four districts after 1993. 

CPRE evaluators used two frame- 
works for these observations. The first 
framework, developed by HRI for the 
ESC initiative, focused on the use of 
inquiry methods in the classroom, with 
particular attention to the design, imple- 
mentation, science content, and equity 
components of the lesson. A second 
"authentic pedagogy" framework was 



based on the work of Ered Newmann 
and Associates (1996) and focused more 
on the interactions between students 
and teachers and teaching for concep- 
tual understanding. This observation 
framework used three general criteria: 
construction of knowledge, disciplined 
inquiry, and value beyond the classroom. 
The results are presented in Analysis 3. 

The HRI Ratings. The HRI classroom 
observation system uses a seven-point 
scale ranging from ineffective instruction 
to exemplary instruction. The score on 
the scale represents a cumulative judg- 
ment based upon the design of the 
lesson, its implementation, subject- 
matter content, and the culture of the 
classroom. The points on the scale are as 
follows: 

1. Ineffective instruction 

2 . Elements of effective instruction 

3 . Beginning stages of effective instruc- 
tion - low 

4. Beginning stages of effective instruc- 
tion - solid 

5 . Beginning stages of effective instruc- 
tion - high 

6. Accomplished, effective instruction 

7. Exemplary instruction 
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Table 6 presents the mean ratings 
from the CPRE observations of three 
groups of teachers: Leader Teachers, 
participants in PTWs, and uninvolved 
teachers. On average, the observed 
lessons of Leader Teachers were judged 
to be slightly below a 4 in 1996-1997 on 
the HRI scale and over a 5 in 1997-1998. 
In contrast, the observed lessons of 
teachers who attended PTWs were 
judged to be about 3.5 in 1996-1997 and 
improved to slightly over 4 in 1997-1998. 
Given that the Leader Teachers were 
nominated for their program and re- 
ceived intensive professional develop- 
ment for three years, this is not surpris- 
ing. Most of the teachers attending 
PTWs in these first two years had re- 
ceived between 32 and 80 hours of 
professional development at the time of 
the observations whereas the Leader 
Teachers had received over 300 hours 
over three previous years. Lessons of 
teachers who had not participated in 
Partnership-sponsored professional 
development observed in 1997-1998 
were judged to be lower in quality than 
those of either of the other groups. 

These results were statistically signifi- 
cant only for Leader Teachers. This may 
be due in part to the small sample sizes, 
particularly for 1997. In the spring of 
1997, CPRE evaluators observed only 35 
science lessons (27 LTI and 8 PTW 
participants). It is worth reiterating 



what these differences mean. They show 
consistent — and in the case of Leader 
Teachers in science, a statistically signifi- 
cant — change in practice from the 
spring of 1997 to the spring of 1998. 
Based on the HRI scale. Leader Teachers 
grew, on average, from a high 3 to a 
solid 5. Peer teachers grew from a solid 3 
to a 4. These growth trends are particu- 
larly notable given the fact that the 
majority of the teachers in the sample 
were observed in both years. These trend 
lines continued into the 1998-1999 
school year. 

Beyond the numbers, the classroom 
observations provided rich glimpses of 
the ways in which teachers were trying 
inquiry-centered instruction. Most of the 
observed lessons incorporated hands-on 
activities designed for student inquiry 
and grouped students for cooperative 
learning. Teachers used class discussions, 
rubrics, and performance activities for 
assessment. CPRE evaluators observed 
many teachers attempting activities that 
would promote deeper student under- 
standing. These teachers implemented 
lessons that encouraged students to 
engage in the scientific process. Most of 
the observed teachers were aware of the 
kinds of activities and prachces that 
promoted higher-order thinking in their 
students, but they often struggled with 
their implementation. Some teachers 
appeared to be using a science achvity 



Table 6. Mean Ratings for Teachers by Type of 
Professional Development, 1996-1997 to 1997-1998 
(with standard deviations in parentheses) 



Category of Teacher 


N 


Average Rating 
1996-1997 


N 


Average Rating 
1997-1998 


Leader Teachers 


27 


3.93 


25 


5.30** 






(1.49) 




(1.35) 


PTW Participants 


8 


3.44 


25 


4.08 






(1.01) 




(1.73) 


Non-participants in Partnership 


NA 


NA 


18 


3.26 


Professionai Deveiopment 








(1.86) 



*** p<.001 ** p<.01 p<.05 
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simply for activity's sake and were not 
able to articulate how the activity would 
enrich students' conceptual understand- 
ing. Yet even these teachers were in the 
process of changing their practice. 

Analysis 3: Authentic Pedagogy 
Ratings. The teachers observed by CPRE 
evaluators were also rated using the 
authentic pedagogy framework. Each 
lesson was examined for the presence of 
the following aspects of instructional 
quality: 

• Higher-order thinking skills. Stu- 
dents manipulate information and 
ideas by synthesizing, generalizing, 
explaining, hypothesizing, or arriv- 
ing at conclusions that produce new 
meaning and understandings for 
them. 

• Substantive conversation. Students 
engage in extended conversational 
exchanges with the teacher and/or 
their peers about subject matter in a 
way that builds an improved and 
shared understanding of ideas or 
topics. 

• Deep knowledge. Instruction ad- 
dresses central ideas of a topic or 
discipline with enough thoroughness 
to explore connections and relation- 
ships and to produce relatively 
complex understandings. 



• Connections to the real world. Stu- 
dents make connections between 
substantive knowledge and either 
public problems or personal experi- 
ences. 

Overall, science teachers who had 
participated in Partnership professional 
development incorporated more higher- 
order thinking, substantive conversation, 
and deeper knowledge in their lessons. 
Table 7 presents observers' ratings on the 
authentic pedagogy scale for science 
lessons conducted by teachers having 
different levels of participation in Part- 
nership-sponsored professional develop- 
ment. The authentic pedagogy ratings 
range from minimal or no use to high 
use on a five-point scale. 

As the data presented in Table 7 
show. Leader Teachers exhibited more 
aspects of authentic pedagogy than did 
either teachers attending PTWs or those 
who had no Partnership professional 
development. There were statistically 
significant differences among the three 
groups in all four areas. Leader Teachers 
had their classes engaged in more 
higher-order thinking than teachers who 
attended PTWs or the non-participants, 
but there was also a significant differ- 
ence between PTW participants and 
non-participants. In classes led by 
Leader Teachers, CPRE evaluators also 
observed more complex conversations 



Table 7. Average Authentic Pedagogy Ratings of Science 
Lessons for Teachers with Varying Amounts of Professional 
Development (with standard deviations in parentheses) 



Type of Professional 
Development 


Higher-order 

Thinking 


Substantive 

Conversation 


Deep 

Knowledge 


Connections to 
the Real World 


LTI (n=20) 


3.40 


3.55 


3.35 


2.79 




(1.30) 


(1.05) 


(1.31) 


(1.18) 


PTW participant in 1996 or 


2.80 


2.72 


2.72 


1.96 


1997 (n=25) 


(1.22) 


(1.14) 


(1.12) 


(.79) 


No MISE Professional 


2.38 


2.00 


2.46 


2.08 


Development (n=13) 


(1.04) 


(1.15) 


(1.13) 


(.95) 
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(deep knowledge) than in those led by 
teachers attending PTWs. However, the 
ratings of both groups were higher than 
those of non-participating teachers. 
Finally, Leader Teachers demonstrated 
significantly more connections to the real 
world than did peer teachers, but were 
no different than non-participants. 

Three major points should be taken 
from this analysis. First, the Leader 
Teachers' classrooms differed signifi- 
cantly from those of all other teachers. In 
the classes of Leader Teachers, there 
were more visible examples of higher- 
order thinking, more evidence of sub- 
stantive conversation between students 
and teachers and among students, and 
more evident attention to the complex 
ideas underlying deep understanding. 
Second, smaller differences were ob- 
served between participants in the PTWs 
and the non-participants. Again, we are 
reminded that the Leader Teachers were 
selected because of their interest (and 
perhaps backgrounds) in science, 
granted special status in their schools, 
and therefore might be expected to be 
more responsive to Partnership profes- 
sional development. And they had 
received much more professional devel- 
opment in science by the spring of 1998 
than teachers who had only attended 
PTWs. 



Analysis 4: Cumulative Impact of 
Partnership Professional Development. 
Evidence has already been presented 
showing the positive relationship be- 
tween participation in Partnership 
professional development and inquiry- 
based teaching. But is there a greater 
impact for those teachers who partici- 
pated in more of the professional devel- 
opment? Is more necessarily better? 
CPRE also examined whether or not 
attending more than one PTW had even 
larger effects on classroom practice. 

In Table 8, the average observational 
ratings of teachers (using the HRI seven- 
point rating scale) who had attended 
two or more PTWs are compared to 
those of teachers who attended only one 
PTW between 1996 and 1999. No dis- 
tinctions were made concerning the 
particular content of workshops teachers 
attended, but rather the hypothesis was 
that attending more PTWs, regardless of 
their content, would increase teachers' 
commitment to and capacity for inquiry- 
centered and standards-based science 
teaching. The results indicate that, on 
average, the 11 teachers who attended 
multiple PTWs had significantly higher 
observational ratings than the 14 who 
attended only one PTW (t=2.07, df=23, 
p=.05). While these results suggest that 



Table 8. Average Ratings for PTW Participants by 
Number of PTW Sessions Attended and the Date of the 
Lost Session (with standard deviations in parentheses) 



Frequency 


Number of Observations 


Average Rating 


One PTW 


14 


3.64 

(1.86) 


Two or more PTWs 


11 


5.00 

(1.41) 


Year of Last PTW 






1997 


10 


4.00 

(1.88) 


1998 


15 


4.40 

(1.76) 
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attending more PTWs might lead to 
higher levels of practice, we cannot 
make attributions of causation here. It 
could be that teachers with higher levels 
of inquiry-based practices were more 
motivated to attend the workshops 
rather than the workshops causing 
higher levels of practice. 

Analysis 5: Analysis of Interview 
Data. In addition to the sample of 
classroom observations, CPRE also 
collected data on instructional practice 
through interviews with teachers, ad- 
ministrators, and district staff. And 
although self-reports about classroom 
practice are not as valid as observations 
(because teachers may be inclined to 
indicate they are doing what they know 
is desirable rather than what they actu- 
ally do), they are a useful complement to 
observations and help place the observa- 
tional data in context. Similarly, when 
survey data are compared over time and 
are found to be consistent with the 
patterns emerging from interviews and 
observations, they strengthen the gener- 
alizations that can be made about the 
trends in classroom practice. 

CPRE evaluators found that most — 
over 90% — teachers attending PTWs 
initially reported their goal was to learn 
to use the new instructional modules 
properly. However, CPRE evaluators 
found that the Leader Teachers and 
those who attended multiple PTWs had 
more ambitious goals that were more 
closely aligned with the Partnership's 
vision for reform. They were focused on 
increasing inquiry-centered questioning 
and investigations in their classrooms. 

As one Leader Teacher reported, 

"...what changed was the focus. We 
were much more inquiry-based. I tried to 
do more questioning... had them do more 
discovery." Examination of the interview 
data collected over time suggest that the 
purpose for science instruction changed 
in many classrooms over the life of the 
Partnership. As a result, each year, more 



students were exposed to science in- 
struction aimed at developing under- 
standing and inquiry skills rather than 
just an interest in science. 

Teachers also reported changing the 
way they conducted lessons in science. 
Over 80% of those interviewed reported 
that they either began to use inquiry or 
that they increased the use of inquiry in 
their classes. One primary teacher 
reported that, in the past, she read a 
book on science to her class and had 
students complete worksheets. Now, she 
sits with the students, presents informa- 
tion, and asks them to explore, observe, 
and predict. This change in practice led 
to "students exploring more... being 
creative in their explaining of 
science... and has enabled students to 
take more ownership [in science be- 
cause] it is more meaningful to them." 
About two-thirds of those interviewed 
reported that they were using some form 
of performance assessment. Many were 
using performance assessments devel- 
oped by teachers with the support of 
MISE. 

Most principals and district staff 
interviewed also reported that teachers' 
practices have changed, and that they 
believed that the changes were deep, 
permanent, and have become part of the 
school fabric. 

Three times between 1994 and 2002, 
CPRE evaluators conducted interviews 
with principals in the four districts. On 
each of these occasions, the vast majority 
of the principals reported being intellec- 
tually and pedagogically excited about 
the Partnership and expressed strong 
commitments to inquiry-based learning. 
They reported that both teacher and 
student interest in science was growing. 
They were effusive, exuberant, and 
inspired by changes in teaching they had 
witnessed in their buildings. Some 
typical comments from elementary 
school principals follow: 
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Children love science. I love seeing 
hands-on, inquiry-based science. It's so 
cool to see the kids in action. And it's 
neat seeing teachers allowing kids to 
investigate and discover. 

...we're now approaching science in a 
whole different way. The old textbook 
approach is out; students think and look 
at things analytically. We know how to 
do an observation and articulate what 
are in those observations. It's a pleasure 
to observe the science lessons. The 
teachers are more knowledgeable about 
content and process. What it does, it 
gives them the tools for how to learn. 
They're getting a philosophy of inquiry. 

There are more children involved in 
science and science is more meaningful. 
The evidence? I'm a former textbook 
learner. I was afraid of science. I wasn't 
excited about science. Now, students are 
excited; they see it as a natural thing. 
It's been a very effective Partnership. A 
lot of teachers have grown. Had it not 
been for MISE, teachers wouldn't have 
tried hands-on...! had to learn to touch 
the worml And pretend that it didn't 
bother me. It's been enlightening. 

The science program has been greatly 
expanded and enhanced. Science is 
emphasized and interesting. It's higher- 
level science. Science has a high degree 
of importance. It's almost equal to other 
areas. Teachers are more comfortable 
teaching science. 

The staff development has refreshed 
teachers. I was an elementary teacher 
and I was afraid of science. Now, it's 
commonplace. They [teachers] have the 
same sense of wonder that the kids 
have! 

When I walk into a classroom now, I see 
a lot of sharing and dialogue and 
excitement and the teacher roving and 
asking higher-level questions and 
having the kids ask, "Let's see what 
happens. " A lot of charts and data 



collections and connections to everyday 
life. I'm seeing evidence of their science 
in the classrooms. Collaborative efforts 
between students. The collaborative 
piece, teaming with teachers, sharing 
discoveries, talking about their findings, 
sharing their scientific methods... 

In addition, CPRE evaluators inter- 
viewed district staff members multiple 
times over the past decade of evaluating 
the Partnership's work. They also re- 
ported profound changes in teacher 
attitudes and teacher classroom practice. 
Describing the shifts in attitudes as 
"cultural changes," one district staff 
member said: 

Teachers are now willing to go to 
meetings without pay. The reward is 
intrinsic not extrinsic. A change has 
occurred in the culture. It's now okay to 
talk about science. 

Another said, "Practices have 
changed. What teachers have learned 
will stay with them." A superintendent 
phrased it more powerfully, "Teachers 
who have been through MISE have 
changed forever." 

District leaders provided examples of 
how teachers' practices have changed as 
a result of their participation in Partner- 
ship activities with MISE. One individual 
described changes in the look of class- 
rooms: 

Kids don't sit in rows facing the teacher. 
They sit in groups of four; there's 
cooperative learning. There's more 
movement in class. Classroom 
management has become [better] 
because there's more movement and 
students feel more comfortable and freer 
to do their work. 

Another individual noted the differ- 
ences in science classes: 

Before, teachers might have done a 
demonstration, kids watched, or all kids 
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did the same science -project. The text is 
not the focus an-ymore. It's inquir-y- 
based. Kids are constructing their own 
lessons. It's hands-on. It's fun for the 
kids. Science didn't come alive [before]. 

A third district individual com- 
mented on the changes in teachers' 
confidence as science teachers as a result 
of their experiences. In turn, teachers' 
increased confidence has led to changes 
in science classes, both in pedagogy and 
curriculum development. She said: 

The amount of teachers' energy and 
their confidence to teach science. They're 
not afraid to touch animals. That's 
great! They've given up some control 
and they're not afraid to do it. 

Yet another district individual com- 
mented on teachers' raised expectations 
for professional development: 

They've [teachers] all become critical 
consumers. Before, it was almost 
expected it [professional development] 
would be boring. You now know what 
good, high-quality professional 
development looks like. You participate, 
you're an active participant. They 
[teachers] demand more now. And 
they've become more vocal. The day of 
accepting just a lecture is almost over. 

Without exception, district leaders 
and staff from all four districts readily 
acknowledged that the Partnership had 
been unprecedented in its commitment, 
quality, duration, scope, financial sup- 
port, and focus. One person said: 

It's the biggest plus for the science 
program ever. The amount of 
professional development, leadership, 
guidance, purchase of materials. It's just 
something we wouldn't have been able 
to do. We would not have had that kind 
of vision. 



This individual's final comment was 
especially telling: "We would not have 
had that kind of vision." While there 
have been many successful school- 
business partnerships, MISE's partner- 
ship with these four school districts has 
been unusual in that one of its stated 
goals has been to challenge the way 
individuals think about instruction, 
learning, science, and professional 
development. 

Summary of Impact on 
Practice 

CPRE analyzed several types of 
evidence about the effects of teacher 
participation in Partnership professional 
development on teachers' instructional 
practice and content knowledge. A 
positive relationship was found between 
participation in professional develop- 
ment (both the LTI and PTWs) and 
inquiry-based instruction in all of the 
analyses. In some analyses, the associa- 
tion between participation in the LTI 
and inquiry-based instruction was 
stronger than the relationship between 
PTW participation and this type of 
instruction. One obvious explanation for 
this difference might be that the partici- 
pants in the PTWs simply had less 
professional development at the time of 
the data collection than the Leader 
Teachers. There may not have been 
sufficient time for the PTWs to have as 
much impact on practice as the LTI. This 
is consistent with other findings showing 
that it takes time for teachers to incorpo- 
rate new techniques into their practice 
(Supovitz, 2001). Another reason could 
be that the LTI directed more attention 
to the development of content knowl- 
edge. A third explanation could be that 
the intense attention given to the Leader 
Teachers over the three years of the LTI 
made them more confident about science 
teaching and the use of inquiry. Linally, 
the difference might be due to selection 
bias; the Leader Teachers were invited to 
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participate because of their interest in 
science and therefore may have been 
engaged in reformed practice before they 
were selected. 

However, in general, the teachers in 
the partner districts reported in both 
surveys and interviews that their teach- 
ing had become more inquiry-based as a 
result of the professional development 
experiences. Independent observers from 
CPRE confirmed the changes among 
teachers who had participated in the LTI 
and the PTWs. Principals and district 
staff also confirmed that Partnership 
professional development had been 
instrumental in altering instruction in 
important ways that increased both 
teacher and student enthusiasm for 
science. 

The Impact on Student 
Outcomes 

Did the students of teachers who 
used more reform-based practices per- 
form better than the students of more 
traditional teachers? Reports from 
interviews and observations and in- 
depth case studies of several schools 
suggest that the nature of student class- 
room experiences and their work dra- 
matically changed over time (Kannapel, 
2003; McVay, 2003a, 2003b; Passantino, 
2003). Students were spending less time 
reading textbooks and memorizing terms 
and definitions and much more time 
doing investigations, preparing lab 
reports, writing in their journals, observ- 
ing demonstrations, and discussing big 
ideas. Everyone agreed that the students 
were gaining a better understanding of 
science and acquiring habits of mind 
associated with scientific work. But 
what about hard data on their perfor- 
mance? CPRE examined results from 
several achievement tests to measure the 
relationship between Partnership profes- 
sional development and student out- 
comes. This report includes the results of 



analyses using three of these tests — the 
SAT-9, the New Jersey state elementary 
and middle school tests (ESPA-Elemen- 
tary School Proficiency Assessment and 
GEPA-Grade Eight Proficiency Assess- 
ment), and a performance assessment 
that was composed of items taken from 
TIMSS (the Third International Math- 
ematics and Science Study). The results 
point to the need for better assessments 
and better assessment systems in science. 

Sources of Data. Information avail- 
able on student outcomes in science in 
the Partnership schools was fragmented 
and flawed. The primary assessment tool 
selected by the Partnership, the SAT-9 
science test, was not well aligned with 
the content of the new K-8 science 
curricula adopted by the districts or with 
the vision of good science teaching 
advocated by the Partnership. In fact, 
given that the science modules were 
often rotated across classrooms so that 
each class of students would have 
covered different units and in a different 
sequence by the time the test was admin- 
istered in the spring, it would have been 
too difficult to achieve alignment with 
any uniform test. Moreover, the SAT-9 
was administered unevenly across the 
schools, and there were no incentives for 
either teachers or students to take it 
seriously. It was simply an additional test 
administered in grades 5 and 7, and was 
often referred to as the CPRE or MISE 
test. Eor all of these reasons, the results 
on the SAT-9 may under-estimate what 
students gained from the reforms. 

The state science assessment used in 
New Jersey in grades 4 and 8 was avail- 
able only in the latter part of the period 
covered by this report. The tests were 
presumably based on the state stan- 
dards, but the test specifications were 
quite broad and the items were never 
released so it was impossible to examine 
their alignment with the content covered 
in the fourth and eighth grades in the 
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partner districts. Hence, we did not have 
true baseline data for using these assess- 
ments to measure the impact of the 
Partnership, nor did we have a basis for 
interpreting the results. We simply 
cannot make inferences about causal 
links between reformed practice and 
student learning given the quality of 
these assessment data. Therefore, we 
frame our conclusions as hypotheses 
based on associations rather than as 
causal statements. 

Analysis 1: Reformed Classroom 
Practice and Student Performance. 

CPRE evaluators constructed multi-level 
models, again using hierarchical linear 
modeling, to examine the relationship 
between teaching practice and student 
achievement. CPRE evaluators con- 
ducted two analyses — one for fifth 
grade and the other for seventh grade. 
The fifth-grade sample consisted of 727 
students in 34 classes in 17 schools; the 
seventh-grade sample included 1,236 
students in 16 classes in 7 schools. These 
were the fifth- and seventh-grade stu- 
dents whose teachers had completed the 
HRI science survey. The outcome vari- 
able in the two models was normed 
student achievement (that is, measured 
in normal curve equivalents) on the SAT- 
9 science assessment. 



Results from these analyses indicated 
a relationship between teaching practice 
and student performance for the fifth 
grade, but not for the seventh grade. In 
the fifth grade, after adjusting for stu- 
dent and teacher background character- 
istics and differences between schools, 
there was a statistically significant 
increase in student performance associ- 
ated with changes in teaching practice. 
An 8% difference in student perfor- 
mance on the SAT-9 assessment was 
found between students with teachers 
using average teaching practice and 
students whose teachers used reformed 
practice. In the seventh grade, no detect- 
able differences were found. 

Figure 4 is a pictorial representation 
of the relationship between standards- 
based teaching practice and student 
performance. After holding constant 
differences in student background, 
teacher background, and schools, fifth- 
grade students, on average, scored 50 on 
the SAT-9. Increased use of reformed 
practice, as reported by teachers, was 
linked to gains in student achievement. 
For the seventh grade, the line is flat, 
meaning that no statistically significant 
impact on performance was associated 
with differences in practice. The lines for 



Figure 4. The Relationship Between Standards-based 
Teaching Practice and Student Performance 
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both grade levels are dashed to show 
that these test scores are derived, not 
actual scores. 

Finally, the qualitative data shed 
some light on these issues. Teachers and 
principals revealed in interviews that 
students' interest in and enthusiasm for 
science were greater than in the past 
when the science curriculum was text- 
book based. A hands-on approach was 
clearly engaging students more and 
making good use of their energies, and 
the inquiry-based curriculum was mak- 
ing them feel like real scientists — form- 
ing hypotheses, thinking critically, and 
using resources to answer their own 
questions. Teachers and students alike 
gave science higher priority. This is 
reflected in the observations of many 
teachers that students often pleaded 
with them to do science every day and 
were disappointed on days when it 
would not be covered. 

Analysis 2: Breadth and Depth of 
Professional Development and Student 
Outcomes. After determining that there 
was a positive relationship between the 
participation of individual fifth-grade 
teachers in Partnership professional 
development and student achievement 
in their classes, CPRE evaluators looked 
at whether school-wide teacher involve- 
ment in the Partnership program was 
related to student outcomes. It was 
hypothesized that as the number of 
teachers in a school participating in the 
Partnership professional development 
increases, the proportion so engaged 
might reach a tipping point after which 
inquiry-centered instruction became the 
norm and even teachers who were not 
participating would also adopt the 
instructional practices, and that this 
level of engagement might affect school- 
wide student achievement. A second 
hypothesis focused on Leader Teachers. 
The argument was that if a small num- 
ber of teachers participated intensely in 
the Partnership professional develop- 



ment, they might become instructional 
leaders in their schools, share their ideas 
and materials, and shape the school- 
wide norms of practice. 

To test these two hypotheses, CPRE 
conducted an analysis of primary teach- 
ers using the results of the third-grade 
performance assessment conducted by 
the Partnership and the results of the 
fourth-grade state science assessment for 
the three New Jersey districts. Measures 
of the percentage of teachers in a school 
who participated in Partnership profes- 
sional development and the amount that 
they had received were used as mea- 
sures of the breadth and depth of the 
saturation of the Partnership's profes- 
sional development. These are measures 
of the degree to which a professional 
community in a school was engaged in 
the professional development. The 
breadth and depth variables were cre- 
ated by aggregating the individual 
teacher participation data to the school 
level. All types of Partnership profes- 
sional development were included — the 
LTI, PTWs, service on instructional 
teams, and participation in assessment 
projects. 

The measure of breadth was the 
percent of teachers in a school who had 
experienced more than one PTW be- 
tween 1996 and 2001. The means for 
this variable in the three New Jersey 
districts were 84.3% for grades K-4 and 
83.5% for grades K-3. Depth was the 
percentage of teachers within a school 
who had participated in more than 200 
hours of Partnership professional devel- 
opment across the five years. The means 
for this indicator in the three New Jersey 
districts were 22.9% for grades K-4 and 
20% for grades K-3. This measure is 
fairly high because the original Leader 
Teachers all participated in 300 hours of 
professional development, and most 
schools had two or three such teachers 
in their primary grades. The depth 
measure can be viewed as an indicator 
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Table 9. Breadth and Depth of Saturation, 
Grades K-4 - New Jersey Districts 



Saturation 

Variable 


Mean 


Minimum 


Maximum 


Standard 

Deviation 


Number of 
Cases 


Breadth 


84.3 


45.5 


100.0 


16.1 


13 


Depth 


22.9 


11.1 


46.0 


10.6 


13 



of the leadership cadre within the 
schools. A school was characterized as 
having high depth if at least 20% of the 
teachers within it had more than 200 
hours of professional development 
during the five years. A school qualified 
as having high breadth if greater than 
84% of teachers (the mean) had at- 
tended at least one professional develop- 
ment workshop from 1996 to 2001. 

In this analysis, CPRE examined the 
relationship between the measures of the 
breadth and depth of professional 
development in the three New Jersey 
districts and the results on the fourth- 
grade state science assessment — the 
New Jersey ESPA (Elementary School 
Proficiency Assessment) — for three 
years and one year of the performance 
assessment scores (third grade). K-4 
teachers in the three New Jersey districts 
were used for the analysis of the fourth- 
grade ESPA results. K-3 teachers in all 
four participating districts were used for 
the analysis of the third-grade perfor- 
mance assessment results. The teachers 
in both samples were current as of the 
2000-2001 school year. Thirteen New 
Jersey schools were included in the first 
analysis and 28 schools from all four 
districts were included in the second 
analysis. Tables 9 and 10 provide the 
results of the breadth and depth analy- 
ses. 

Tables 9 and 10 reveal that the 
breadth of participation in professional 
development was positively related to 
ESPA scores, but it was not strongly 
related to the performance assessment 
scores. The depth measure was not a 
good predictor of improved perfor- 
mance. It was not strongly related to the 



1999 ESPA scores, 2001 ESPA scores, or 
the performance assessment scores, and 
was negatively correlated with the 2000 
ESPA scores. Since the depth measure 
was a poor predictor, it did not make 
sense to combine it with the breadth 
measure. Breadth of participation ap- 
peared to be the aspect of saturation in a 
school that was related to higher perfor- 
mance. The analysis, then, focused on 
breadth and depth individually. 

Because the number of cases was 
small, it was hard for statistical esti- 
mates, such as correlations, to reach 
significance levels. However, since the 
Partnership was engaged in district- 
wide reform, we were looking at the 
entire population of K-4 teachers within 
the three New Jersey districts (for the 
ESPA scores) and the entire population 
of K-3 teachers within all four Partner- 
ship districts (for the performance 
assessment analysis). So we concluded 
that these estimates were true for these 
populations, regardless of their signifi- 
cance levels. However, we could not 
reach conclusions that extend beyond 
these specific populations. 

The next step was to examine 
whether the relationship between inde- 
pendent and dependent variables was 
curvilinear. This was done only for the 
breadth variable, since there was not a 
substantial relationship between the 
depth measure and school mean scores. 
The results showed that the relationship 
between broad participation in profes- 
sional development and the 2001 ESPA 
scores was curvilinear. Pigure 5 shows 
that a curvilinear model fits the data 
better and explains more of the test score 
variation. The effect of the breadth 
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Table 1 0. Breadth and Depth of Saturation, 
Grades K-3 - All Districts 



Saturation 

Variable 


Mean 


Minimum 


Maximum 


Standard 

Deviation 


Number of 
Cases 


Breadth 


83.5 


31.0 


100.0 


17.7 


28 


Depth 


20.0 


0 


62.5 


14.8 


28 



variable was fairly flat until 78% or more 
of the teachers in a school were engaged 
in Partnership professional development. 
The relationship was positive from that 
point forward except for one case. This 
suggests that the tipping point for in- 
structional reform might be much higher 
— at least in science — than is usually 
assumed by school reformers. A rather 
high proportion of teachers had to be 
engaged before positive effects on stu- 
dent achievement were observed on the 
results of the 2001 ESPA. However, the 
results of the analysis were mixed as the 
relationship between the breadth vari- 
able and the 1999 ESPA scores was not 
curvilinear. 



The results shown in Table 11 indicate 
that the breadth of teacher participation 
in professional development was posi- 
tively related to the 1999 and 2001 ESPA 
science scores. The relationship between 
the 2000 ESPA score and breadth was not 
substantial. There is a curvilinear relation- 
ship between breadth of professional 
development and the 2001 ESPA scores. 
This relationship is fairly flat until about 
78% of a school's faculty has experienced 
at least one form of professional develop- 
ment. While breadth appears to be related 
to higher scores, the depth of participa- 
tion does not. 



Figure 5. Elementary School Proficiency 
Assessment 2001 General Education School Mean 
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Table 1 1 . Zero-order Correlations Between Breadth and Depth of 
Professional Development and Science Assessment Results 







ESPA School Means 




Performance Assessment 
Scores 


Test date 


1999 


2000 


2001 


2000-2001 school year 


Breadth 


.261 


.023 


.398 


-.032 


Depth 


-.058 


-.314 


.013 


.036 


Number of cases 




13* 




28 



* Total number of cases across all three years. 



These data suggest that achieving 
broad faculty participation in profes- 
sional development may be more impor- 
tant than providing intense experiences 
of some teachers. In the Partnership 
schools, having a large number of teach- 
ers who have had some professional 
development is more strongly associated 
with student performance than having a 
small number of teachers who have been 
deeply engaged in professional develop- 
ment. On the one hand, these findings 
make common sense. If more teachers 
are receiving training, more will be using 
inquiry methods in their classrooms, and 
more students will be exposed to inquiry 
science. The findings are also consistent 
with studies of professional community 
that have shown that having a shared 
vision of good practice and a common 
language to discuss practice facilitate the 
improvement of classroom practice. 

Also, they are consistent with earlier 
CPRE findings about the effectiveness of 
Leader Teachers in spreading inquiry 
methods within their schools. Some of 
the Leader Teacher teams were unable to 
have much impact on the practices of 
their colleagues because they lacked the 
opportunity to work with them and the 
creation of such opportunities were 
obstructed by norms of teacher au- 
tonomy and privacy, competing instruc- 
tional priorities, schedules, and lack of 
principal support. 

The findings are encouraging be- 
cause they are consistent with MISE's 
theory of action. The provision of broad 
access to professional development 



combined with on-site support contrib- 
uted to improved science instruction, 
which in turn contributed to better 
student outcomes. While not definitive, 
this analysis suggests that more refined 
work using better-aligned outcome 
measures, student-level performance 
data, and better measures of school 
conditions might reveal even stronger 
relationships. 

Summary: The Impact of 
Professional Development 
on Student Outcomes 

Although hampered by inadequate 
measures of student performance in 
science, CPRE conducted two analyses 
of the impact of the Partnership's work 
on student outcomes. The first examined 
the relationship between teacher use of 
reform-based classroom practice and 
student achievement in science and the 
other examined the relationship between 
levels of participation in MISE profes- 
sional development in schools and 
school-wide student achievement. In the 
first study, which used hierarchical 
linear modeling, a statistically significant 
relationship between reform-based 
teaching practice and student achieve- 
ment was found in the fifth grade, but 
not the seventh grade. In the second 
study, CPRE analyzed the breadth 
versus depth of participation in MISE 
professional development. The main 
finding was that while breadth of 
teacher participation in the New Jersey 
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Partnership schools was related to 
higher student scores on the New Jersey 
state test (Elementary Science Perfor- 
mance Assessment), the depth of their 
participation was not. These results 
suggest that reaching a broader audi- 
ence of teachers may help spread the 
reform vision and impact student learn- 
ing more than helping a small group of 
teachers achieve deep understanding. Of 
course, MISE and the Partnership did 
both, and the interactions between the 
capacity and commitment of the Leader 
Teachers and the breadth and depth of 
the participation in the PTWs are not 
clear. It may be that broad participation 
helps build a learning community com- 
mitted to the reform and produces the 
social pressures and supports needed for 
sustained change. Clearly, a small group 
of trained teachers can choose to keep 
new knowledge about content and 
pedagogy to themselves or they can 
share it with colleagues and promote the 
growth of others. Which they choose to 
do may depend on the nature of a 
school's professional community and on 
its leadership. 

Lessons and Challenges 

What lessons can be drawn from the 
experience of MISE and the Partnership? 
It seems to us that there are some impor- 
tant ones for those who want to improve 
the quality of teaching, those who plan 
professional development, and leaders of 
reform support organizations that work 
with schools. In our view, these lessons 
are: 

1 . The importance of being both sys- 
temic and specific. MISE took a sys- 
temic approach to reform in the 
partner districts, but they also real- 
ized that teachers needed a concrete 
vision of good science teaching, good 
instructional materials, and acces- 
sible, practical professional develop- 
ment. They did not fall into the trap 
of believing that curriculum stan- 



dards, aligned policies, and account- 
ability procedures would be suffi- 
cient to bring about the desired 
changes in classroom practice. Their 
specificity about high-quality science 
education that enabled them to work 
with their partners to build a solid 
science curriculum and design 
professional development whose 
utility was recognized and embraced 
by the vast majority of teachers. In 
addition, the development and 
recognition of Leader Teachers and 
the engagement of district staff in the 
work made the Partnership's profes- 
sional development more compelling 
and more powerful than it would 
have been otherwise. Neither align- 
ing local policies with content stan- 
dards nor providing standards-based 
professional development probably 
would have produced the same 
results on their own. The combina- 
tion led to the development of dis- 
trict and school cultures and con- 
texts in which the professional 
development was useful, valued, and 
shared. 

2 . The effectiveness of a comprehensive 
approach to professional develop- 
ment. The general professional 
development model used by the 
Partnership combined extended 
immersion in curriculum content 
and pedagogy with strong follow-up 
support. Over time, this general 
model for teacher learning led to 
significant changes in classroom 
practice. Their experience shows that 
the combination of off-site immersion 
in content and pedagogy and on-site 
support and reflection is a powerful 
strategy for promoting teacher 
learning and changing classroom 
practice. 

3. The importance of offering teachers 
recurrent opportunities to participate 
in professional development. If the 
professional development is to be 
practical, it must be focused on 
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specific chunks of curriculum. If it is 
to be effective, it must be intense 
enough for teachers to get deeply 
engaged in the subject matter and 
pedagogical issues surrounding it. 
Therefore, to have an impact on the 
entire curriculum, teachers must 
have repeated opportunities to 
participate over a relatively long 
period of time. In this case, each 
summer, they could take one or even 
two new PTWs and over three or 
four summers, they would have 
covered the backbone of the curricu- 
lum at their grade level. 

4 . The enormous benefits of developing, 
using, and respecting Leader Teach- 
ers. The development of teacher 
leadership through the LTI and the 
instructional teams was another key 
to MISE's success. These teachers not 
only provided on-site support for 
other teachers attempting to use 
inquiry-centered approaches, they 
also recruited colleagues into the 
PTWs and built communities of 
practice in their schools that sus- 
tained teacher involvement and 
began a process of continuous im- 
provement through dialogue, reflec- 
tion, and collaboration. The develop- 
ment and use of Leader Teachers not 
only built local capacity to design 
and provide professional develop- 
ment, but also altered district and 
school cultures by redefining the 
roles available to teachers and gener- 
ating new respect for their expertise. 

5 . The importance of building princi- 
pals. CPRE's analyses show that the 
understanding and support of 
principals influenced the impact of 
professional development on class- 
room practice and teachers' roles. 
This was true for the Leader Teach- 
ers and for those who participated in 
the PTWs. MISE initially involved the 
principals in the Leader Teacher 
program and then neglected them 
for several years. Eeedback from 



their evaluation alerted them to this 
problem, and they responded with 
principals' institutes that helped 
principals understand the vision, 
gave them tools to monitor its enact- 
ment and spread, and helped princi- 
pals recognize the need to use 
teacher expertise to build effective 
communities of practice. 

6 . The importance of persistence over 
time. MISE invested in a long-term 
partnership with the four districts 
that allowed for the development of 
shared understandings and a com- 
mon culture that sustained the work. 
The PTWs and the Partnership's 
follow-up support provided time for 
teachers to master new content, 
integrate new strategies into their 
practice, reflect on the responses 
they were getting from their stu- 
dents, and engage in dialogue with 
others who were trying similar 
approaches. Teachers were able to 
attend PTWs each summer if they 
wished and the data show strong 
relationships both between their 
extended engagement in learning 
and the changes they made in their 
practice. It takes considerable time 
and support to make inquiry- 
centered instruction the norm in 
classrooms. Teachers showed the 
greatest change in practice after 
three years of participation and more 
than 80 hours of professional devel- 
opment. Prior to that tipping point, 
most teachers made only incremental 
changes in their practice. 

7. Using tools as mechanisms for trans- 
ferring new knowledge. MISE's 
vision of inquiry-centered instruction 
guided its work, but it was the 
development and selection of tools 
that embodied the vision — curricu- 
lum frameworks, science modules, 
rubrics, performance assessments, 
and observation instruments — that 
allowed teachers and administrators 
to transform the vision into practice. 
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8. The importance of reinforcing re- 
formed practices in the workplace. 
MISE's work shows that high-quality, 
high-utility professional development 
can attract teachers and that many 
of them will continue to participate 
over time. Volunteerism generally 
works, but not all teachers will 
participate in enough summer insti- 
tutes to reach a tipping point in their 
practice. This observation, along 
with the finding that a relatively 
high proportion of teachers had to 
participate before school-wide effects 
on performance were realized, points 
to the need for robust, on-site follow- 
up support. 

9. The relationship between coherence, 
sustainability, and impact. There 
was considerable turnover in the 
four districts during the period 
described in this report. Staff mobil- 
ity and turnover mean that access to 
curriculum-based professional 
development has to be regular and 
ongoing to have powerful effects on 
practice. By aligning their policies 
around a specific vision of good 
practice, offering a menu of work- 
shops that were periodically re- 
peated, and providing new teachers 
with orientations to inquiry-centered 
instruction, the districts were able to 
develop and sustain particular 
norms of good practice. 

10. The importance of breadth as well as 
depth in the design of reform 
strategies. One of the strengths of 
the LTI was that it concentrated on 
increasing teachers' knowledge of 
science and skill at designing and 
managing student inquiry. The 
strength of the PTWs was that they 
reached a wide audience, so that 
many teachers in a school experi- 
enced similar professional develop- 
ment and shared a common vision 
of good practice. Both gave teachers 
a common experience from which 
they built professional communities. 



While the notion that depth is 
preferred to breadth is a popular 
idea — less is more — the evidence 
suggests that this argument may not 
apply to school change. It was broad 
participation in professional devel- 
opment that was shown to be 
positively related to school-wide 
student achievement, not the depth 
of the experiences of Leader Teach- 
ers. While it is no doubt true that the 
best outcomes would be achieved by 
a program that provided both deep 
engagement in the content and 
broad participation, the evidence 
should raise a red flag for those who 
design programs for small numbers 
of teachers and hope that the 
knowledge they gain will be trans- 
ferred to others. 

11. The importance of seeking and 
using evaluative feedback. Many 
organizations hire evaluators, but 
only some make use of them. MISE 
has been, and continues to be, a 
learning organization. MISE has 
supported evaluation of its work 
and paid close attention to the 
results. Over the years, CERE evalu- 
ators have provided regular feed- 
back on the professional develop- 
ment provided by MISE and the 
Partnership and annually offered 
recommendations for the improve- 
ment of the entire initiative. MISE 
staff have listened to this feedback, 
reflected on it, and typically made 
modifications to their approach. 
Most of the recommendations 
offered in CPRE's annual reports 
have resulted in MISE taking 
action. The title of this report, 
"Getting It Right," reflects this 
commitment to continuous improve- 
ment. 

Challenges and Issues 

Challenges are simply lessons of a 

different kind, pointing to problems that 

must be addressed, pitfalls to avoid, or 
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obstacles to be overcome. The Partner- 
ship certainly faced some challenges, 
and others can benefit from its experi- 
ence. 

1 . First, there is the very important 
problem of identifying measures of 
student learning that are sensitive to 
the outcomes being sought by the 
reforms. Here, the principal investi- 
gator of the CPRE evaluation team 
must take responsibility for provid- 
ing bad advice. In retrospect, it 
would have been better to have used 
some form of student work sampling 
or to have developed some common 
assessments for use with a sample of 
key science modules. Either strategy 
would have provided MISE and the 
Partnership with useful measures of 
progress over time and supported 
studies of the relationships among 
participation in the professional 
development, teaching practices, 
and student performance. It also 
would have permitted evaluation of 
the effectiveness of the modules 
themselves. Instead, concerned about 
possible negative reactions from 
teachers worried about the use of 
student assessment data to assess 
their teaching performance, CPRE 
recommended use of the SAT-9 and 
the state science assessment. Neither 
of these measures proved to be well- 
aligned with the changes taking 
place in teaching and learning. As a 
result, the evidence available on the 
impact of Partnership professional 
development on student learning is 
limited, and we cannot adequately 
determine how much impact the 
reforms in teaching actually had on 
student performance. Clearly, the 
standardized tests were not aligned 
with inquiry-centered instruction or 
the science content being taught, and 
some of the most important effects of 
inquiry-centered teaching were not 
captured by these measures as they 
are not designed to assess students' 



understanding of the scientific pro- 
cess or their willingness to engage in 
observation, analysis, and explana- 
tion. Data from interviews, class- 
room observations, and teacher sur- 
veys all suggest that there have been 
significant effects on student perfor- 
mance, but this is less persuasive 
than having robust and independent 
measures. It will be important for 
future studies to use measures that 
are more closely aligned with the 
reforms being sought in science 
teaching. 

2 . A similar problem arises from the 
lack of appropriate and acceptable 
measures of teachers' knowledge of 
the subject matter. While tests of 
science knowledge exist, there are 
not good measures of pedagogical 
content knowledge available. Such 
instruments would allow for the 
customization of professional devel- 
opment to meet the needs of indi- 
vidual teachers and improve the 
overall design of learning opportuni- 
ties for teachers. We also say accept- 
able because the problem is broader 
than the lack of adequate measures; 
it also involves the willingness of 
teachers to subject themselves to tests 
of their content knowledge. While 
the reluctance of experienced teach- 
ers to take subject-matter tests is 
understandable, it obstructs the 
customization of professional devel- 
opment and means that knowledge- 
able and skilled teachers have to be 
subjected to professional develop- 
ment curricula designed for teachers 
who are less well prepared. 

3. Variations in the science modules in 
the four Partnership districts and the 
practice of rotating science modules 
across classrooms in several of the 
districts made it difficult to build 
professional learning communities in 
the schools, hampered the design of 
follow-up procedures, and limited 
the ability of the Partnership to use 
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review of student work as a vehicle 
for teacher learning. The Partnership 
would have been more effective if all 
of the districts had adopted the same 
science units at the same grade 
levels, and if all teachers had used 
them in the same sequence and at 
roughly the same time. This would 
have permitted the follow-up sup- 
port to focus more attention on 
student work and would have 
allowed more cross-school network- 
ing and benchmarking. 

4 . While the visibility and status of 
science education was raised enor- 
mously in the four districts, it was a 
constant struggle in some schools to 
maintain a focus on science. The 
attention directed to the results of 
state assessments in literacy and 
mathematics by some school princi- 
pals was a perpetual threat to the 
Partnership's work. Even the intro- 
duction of a state assessment in 
science in New Jersey did not solve 
this problem as some principals 
adhered to the view that literacy and 
mathematics were foundational 
subjects and science was not. Yet, 
science provides an intellectual 
framework and a perspective on the 
world that also is foundational. MISE 
attempted to address this by design- 
ing PTWs on science and literacy 
that helped teachers address literacy 
skills through materials containing 
scientific content. But the problem of 
keeping the quality of science teach- 
ing, and time for inquiry, on the 
agenda is a continuing challenge. 

These challenges are important, and 
should be addressed, but they should not 
overshadow what MISE has accom- 
plished in the past decade. MISE and the 
Partnership have provided hundreds of 
teachers in the four partner districts 
with high-quality professional develop- 
ment for almost a decade. They have 
improved teaching and learning in 
science, and indirectly in other subjects, 
in hundreds of classrooms. They have 



demonstrated that the capacities and 
cultures of local schools and districts can 
be changed in fundamental ways. The 
quality and impact of the professional 
development experiences provided has 
remained high even as the responsibility 
for planning and delivering them has 
been shifted to Leader Teachers and 
district staff. Science teaching in the four 
partner districts is fundamentally differ- 
ent now from what it was when MISE 
began its work. MISE's theory of action 
has yielded good results, and because it 
continues to work to "get it right" by 
listening, collaborating, and redesigning, 
it is likely to yield even better results in 
the future. 
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